Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/with_whim_auc_crest_weights/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/with_whim_auc_crest_weights/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 4,
 'device': device(type='cuda', index=0),
 'dropout': 0.0,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/train_whim_mean_crest_weights.npz',
                   '/home/saxelrod/chemprop_cov_2/features/whim/train_whim_std_crest_weights.npz'],
 'features_scaling': True,
 'features_size': None,
 'ffn_hidden_size': 400,
 'ffn_num_layers': 1,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'grad_clip': None,
 'hidden_size': 400,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': False,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/with_whim_auc_crest_weights',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/test_whim_mean_crest_weights.npz',
                                 '/home/saxelrod/chemprop_cov_2/features/whim/test_whim_std_crest_weights.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/val_whim_mean_crest_weights.npz',
                                '/home/saxelrod/chemprop_cov_2/features/whim/val_whim_std_crest_weights.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=400, bias=False)
        (W_h): Linear(in_features=400, out_features=400, bias=False)
        (W_o): Linear(in_features=533, out_features=400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=628, out_features=1, bias=True)
  )
)
Number of parameters = 433,029
Moving model to cuda
Epoch 0
Validation auc = 0.394836
Epoch 1
Validation auc = 0.415493
Epoch 2
Validation auc = 0.439437
Epoch 3
Validation auc = 0.462441
Epoch 4
Loss = 7.5271e-01, PNorm = 32.8335, GNorm = 2.1233, lr_0 = 6.5000e-04
Validation auc = 0.484977
Epoch 5
Validation auc = 0.542723
Epoch 6
Validation auc = 0.553991
Epoch 7
Validation auc = 0.589671
Epoch 8
Validation auc = 0.587793
Epoch 9
Loss = 5.6684e-01, PNorm = 32.9141, GNorm = 2.8700, lr_0 = 9.9743e-04
Validation auc = 0.646948
Epoch 10
Validation auc = 0.669953
Epoch 11
Validation auc = 0.610329
Epoch 12
Validation auc = 0.593897
Epoch 13
Validation auc = 0.596714
Epoch 14
Loss = 5.6161e-01, PNorm = 32.9844, GNorm = 1.8263, lr_0 = 9.8890e-04
Validation auc = 0.610329
Epoch 15
Validation auc = 0.623944
Epoch 16
Validation auc = 0.633333
Epoch 17
Validation auc = 0.650235
Epoch 18
Validation auc = 0.645070
Epoch 19
Loss = 5.3612e-01, PNorm = 33.0716, GNorm = 2.0901, lr_0 = 9.8045e-04
Validation auc = 0.647887
Epoch 20
Validation auc = 0.641784
Epoch 21
Validation auc = 0.640376
Epoch 22
Validation auc = 0.638028
Epoch 23
Validation auc = 0.656338
Epoch 24
Loss = 3.9573e-01, PNorm = 33.1725, GNorm = 0.7509, lr_0 = 9.7207e-04
Validation auc = 0.648826
Epoch 25
Validation auc = 0.695775
Epoch 26
Validation auc = 0.705634
Epoch 27
Validation auc = 0.674648
Epoch 28
Validation auc = 0.696244
Epoch 29
Loss = 4.4956e-01, PNorm = 33.2794, GNorm = 3.7484, lr_0 = 9.6376e-04
Validation auc = 0.672770
Epoch 30
Validation auc = 0.667606
Epoch 31
Validation auc = 0.703756
Epoch 32
Validation auc = 0.671362
Epoch 33
Validation auc = 0.646479
Epoch 34
Loss = 4.1900e-01, PNorm = 33.3958, GNorm = 1.9799, lr_0 = 9.5552e-04
Validation auc = 0.665728
Epoch 35
Validation auc = 0.658216
Epoch 36
Validation auc = 0.659624
Epoch 37
Validation auc = 0.699531
Epoch 38
Validation auc = 0.688732
Epoch 39
Loss = 3.8827e-01, PNorm = 33.5349, GNorm = 2.8191, lr_0 = 9.4735e-04
Validation auc = 0.712676
Epoch 40
Validation auc = 0.723944
Epoch 41
Validation auc = 0.684977
Epoch 42
Validation auc = 0.690610
Epoch 43
Validation auc = 0.686385
Epoch 44
Loss = 3.3914e-01, PNorm = 33.6776, GNorm = 2.6127, lr_0 = 9.3925e-04
Validation auc = 0.672300
Epoch 45
Validation auc = 0.723005
Epoch 46
Validation auc = 0.702817
Epoch 47
Validation auc = 0.692019
Epoch 48
Validation auc = 0.707981
Epoch 49
Loss = 3.6103e-01, PNorm = 33.8236, GNorm = 1.5564, lr_0 = 9.3122e-04
Validation auc = 0.676056
Epoch 50
Validation auc = 0.654460
Epoch 51
Validation auc = 0.663850
Epoch 52
Validation auc = 0.678404
Epoch 53
Validation auc = 0.676526
Epoch 54
Loss = 3.3523e-01, PNorm = 33.9836, GNorm = 0.7435, lr_0 = 9.2326e-04
Validation auc = 0.690610
Epoch 55
Validation auc = 0.680751
Epoch 56
Validation auc = 0.700939
Epoch 57
Validation auc = 0.668545
Epoch 58
Validation auc = 0.672770
Epoch 59
Loss = 2.3542e-01, PNorm = 34.1603, GNorm = 2.8283, lr_0 = 9.1537e-04
Validation auc = 0.659155
Epoch 60
Validation auc = 0.662441
Epoch 61
Validation auc = 0.666197
Epoch 62
Validation auc = 0.689671
Epoch 63
Validation auc = 0.700000
Epoch 64
Loss = 1.9751e-01, PNorm = 34.3435, GNorm = 2.4814, lr_0 = 9.0754e-04
Validation auc = 0.678404
Epoch 65
Validation auc = 0.686385
Epoch 66
Validation auc = 0.681221
Epoch 67
Validation auc = 0.711268
Epoch 68
Validation auc = 0.707512
Epoch 69
Loss = 1.6655e-01, PNorm = 34.5170, GNorm = 2.5711, lr_0 = 8.9978e-04
Validation auc = 0.673709
Epoch 70
Validation auc = 0.672300
Epoch 71
Validation auc = 0.659624
Epoch 72
Validation auc = 0.692958
Epoch 73
Validation auc = 0.672300
Epoch 74
Loss = 1.9009e-01, PNorm = 34.6601, GNorm = 1.8924, lr_0 = 8.9209e-04
Validation auc = 0.666667
Epoch 75
Validation auc = 0.692958
Epoch 76
Validation auc = 0.673239
Epoch 77
Validation auc = 0.661033
Epoch 78
Validation auc = 0.691080
Epoch 79
Loss = 1.9257e-01, PNorm = 34.8112, GNorm = 3.6890, lr_0 = 8.8447e-04
Validation auc = 0.691080
Epoch 80
Validation auc = 0.674178
Epoch 81
Validation auc = 0.682629
Epoch 82
Validation auc = 0.698592
Epoch 83
Validation auc = 0.673239
Epoch 84
Loss = 1.3804e-01, PNorm = 34.9489, GNorm = 1.1411, lr_0 = 8.7691e-04
Validation auc = 0.668545
Epoch 85
Validation auc = 0.681221
Epoch 86
Validation auc = 0.674178
Epoch 87
Validation auc = 0.663380
Epoch 88
Validation auc = 0.690610
Epoch 89
Loss = 1.7787e-01, PNorm = 35.0838, GNorm = 1.2868, lr_0 = 8.6941e-04
Validation auc = 0.680751
Epoch 90
Validation auc = 0.666667
Epoch 91
Validation auc = 0.653052
Epoch 92
Validation auc = 0.660094
Epoch 93
Validation auc = 0.649765
Epoch 94
Loss = 1.5271e-01, PNorm = 35.2033, GNorm = 1.7333, lr_0 = 8.6198e-04
Validation auc = 0.687793
Epoch 95
Validation auc = 0.696714
Epoch 96
Validation auc = 0.674178
Epoch 97
Validation auc = 0.655399
Epoch 98
Validation auc = 0.648357
Epoch 99
Loss = 1.0768e-01, PNorm = 35.3099, GNorm = 0.6621, lr_0 = 8.5461e-04
Validation auc = 0.656808
Epoch 100
Validation auc = 0.659624
Epoch 101
Validation auc = 0.682160
Epoch 102
Validation auc = 0.694836
Epoch 103
Validation auc = 0.683099
Epoch 104
Loss = 6.8815e-02, PNorm = 35.4203, GNorm = 0.7029, lr_0 = 8.4730e-04
Validation auc = 0.678404
Epoch 105
Validation auc = 0.673709
Epoch 106
Validation auc = 0.669484
Epoch 107
Validation auc = 0.688732
Epoch 108
Validation auc = 0.696714
Epoch 109
Loss = 8.8567e-02, PNorm = 35.5191, GNorm = 1.2770, lr_0 = 8.4006e-04
Validation auc = 0.683099
Epoch 110
Validation auc = 0.693897
Epoch 111
Validation auc = 0.658685
Epoch 112
Validation auc = 0.679812
Epoch 113
Validation auc = 0.656808
Epoch 114
Loss = 1.5420e-01, PNorm = 35.6148, GNorm = 3.5346, lr_0 = 8.3288e-04
Validation auc = 0.686854
Epoch 115
Validation auc = 0.700939
Epoch 116
Validation auc = 0.658685
Epoch 117
Validation auc = 0.660563
Epoch 118
Validation auc = 0.671831
Epoch 119
Loss = 1.2198e-01, PNorm = 35.7076, GNorm = 0.9361, lr_0 = 8.2576e-04
Validation auc = 0.660563
Epoch 120
Validation auc = 0.668545
Epoch 121
Validation auc = 0.692488
Epoch 122
Validation auc = 0.697183
Epoch 123
Validation auc = 0.683568
Epoch 124
Loss = 1.3483e-01, PNorm = 35.8028, GNorm = 1.4244, lr_0 = 8.1870e-04
Validation auc = 0.684038
Epoch 125
Validation auc = 0.661502
Epoch 126
Validation auc = 0.664789
Epoch 127
Validation auc = 0.703286
Epoch 128
Validation auc = 0.671831
Epoch 129
Loss = 9.4919e-02, PNorm = 35.8863, GNorm = 1.9494, lr_0 = 8.1170e-04
Validation auc = 0.664319
Epoch 130
Validation auc = 0.677934
Epoch 131
Validation auc = 0.685446
Epoch 132
Validation auc = 0.676995
Epoch 133
Validation auc = 0.683099
Epoch 134
Loss = 7.3235e-02, PNorm = 35.9886, GNorm = 0.9593, lr_0 = 8.0476e-04
Validation auc = 0.690141
Epoch 135
Validation auc = 0.691080
Epoch 136
Validation auc = 0.684507
Epoch 137
Validation auc = 0.678873
Epoch 138
Validation auc = 0.684038
Epoch 139
Loss = 6.0190e-02, PNorm = 36.0763, GNorm = 0.9320, lr_0 = 7.9788e-04
Validation auc = 0.679812
Epoch 140
Validation auc = 0.673239
Epoch 141
Validation auc = 0.665258
Epoch 142
Validation auc = 0.672770
Epoch 143
Validation auc = 0.671362
Epoch 144
Loss = 4.4301e-02, PNorm = 36.1468, GNorm = 0.7298, lr_0 = 7.9106e-04
Validation auc = 0.660563
Epoch 145
Validation auc = 0.658216
Epoch 146
Validation auc = 0.673709
Epoch 147
Validation auc = 0.673239
Epoch 148
Validation auc = 0.652582
Epoch 149
Loss = 5.1635e-02, PNorm = 36.2082, GNorm = 1.9075, lr_0 = 7.8430e-04
Validation auc = 0.649765
Epoch 150
Validation auc = 0.681221
Epoch 151
Validation auc = 0.676526
Epoch 152
Validation auc = 0.665728
Epoch 153
Validation auc = 0.684038
Epoch 154
Loss = 7.5417e-02, PNorm = 36.2730, GNorm = 1.9090, lr_0 = 7.7759e-04
Validation auc = 0.700939
Epoch 155
Validation auc = 0.653991
Epoch 156
Validation auc = 0.636620
Epoch 157
Validation auc = 0.661972
Epoch 158
Validation auc = 0.670423
Epoch 159
Loss = 4.6595e-02, PNorm = 36.3383, GNorm = 0.5916, lr_0 = 7.7095e-04
Validation auc = 0.671362
Epoch 160
Validation auc = 0.675587
Epoch 161
Validation auc = 0.686385
Epoch 162
Validation auc = 0.684977
Epoch 163
Validation auc = 0.678873
Epoch 164
Loss = 9.0508e-02, PNorm = 36.3956, GNorm = 0.5608, lr_0 = 7.6436e-04
Validation auc = 0.669484
Epoch 165
Validation auc = 0.673239
Epoch 166
Validation auc = 0.667606
Epoch 167
Validation auc = 0.660094
Epoch 168
Validation auc = 0.665258
Epoch 169
Loss = 3.2718e-02, PNorm = 36.4539, GNorm = 0.6118, lr_0 = 7.5782e-04
Validation auc = 0.673709
Epoch 170
Validation auc = 0.673709
Epoch 171
Validation auc = 0.667606
Epoch 172
Validation auc = 0.668075
Epoch 173
Validation auc = 0.666667
Epoch 174
Loss = 1.0664e-01, PNorm = 36.5065, GNorm = 1.7402, lr_0 = 7.5134e-04
Validation auc = 0.681690
Epoch 175
Validation auc = 0.676526
Epoch 176
Validation auc = 0.683099
Epoch 177
Validation auc = 0.684977
Epoch 178
Validation auc = 0.670892
Epoch 179
Loss = 5.2012e-02, PNorm = 36.5632, GNorm = 2.0982, lr_0 = 7.4492e-04
Validation auc = 0.673239
Epoch 180
Validation auc = 0.681690
Epoch 181
Validation auc = 0.666667
Epoch 182
Validation auc = 0.653052
Epoch 183
Validation auc = 0.655869
Epoch 184
Loss = 2.1303e-02, PNorm = 36.6194, GNorm = 0.2384, lr_0 = 7.3855e-04
Validation auc = 0.665728
Epoch 185
Validation auc = 0.672300
Epoch 186
Validation auc = 0.664319
Epoch 187
Validation auc = 0.661033
Epoch 188
Validation auc = 0.665728
Epoch 189
Loss = 2.1974e-02, PNorm = 36.6677, GNorm = 0.2346, lr_0 = 7.3224e-04
Validation auc = 0.675117
Epoch 190
Validation auc = 0.689671
Epoch 191
Validation auc = 0.691080
Epoch 192
Validation auc = 0.677934
Epoch 193
Validation auc = 0.679812
Epoch 194
Loss = 3.7949e-02, PNorm = 36.7188, GNorm = 0.9818, lr_0 = 7.2598e-04
Validation auc = 0.682160
Epoch 195
Validation auc = 0.658685
Epoch 196
Validation auc = 0.652582
Epoch 197
Validation auc = 0.666197
Epoch 198
Validation auc = 0.652582
Epoch 199
Loss = 5.6394e-02, PNorm = 36.7724, GNorm = 1.4279, lr_0 = 7.1977e-04
Validation auc = 0.646479
Epoch 200
Validation auc = 0.666197
Epoch 201
Validation auc = 0.678873
Epoch 202
Validation auc = 0.662911
Epoch 203
Validation auc = 0.658685
Epoch 204
Loss = 1.9408e-02, PNorm = 36.8344, GNorm = 0.5168, lr_0 = 7.1362e-04
Validation auc = 0.667136
Epoch 205
Validation auc = 0.666667
Epoch 206
Validation auc = 0.677934
Epoch 207
Validation auc = 0.681221
Epoch 208
Validation auc = 0.684038
Epoch 209
Loss = 2.9068e-02, PNorm = 36.8895, GNorm = 0.2780, lr_0 = 7.0752e-04
Validation auc = 0.679343
Epoch 210
Validation auc = 0.678404
Epoch 211
Validation auc = 0.679812
Epoch 212
Validation auc = 0.676526
Epoch 213
Validation auc = 0.678873
Epoch 214
Loss = 1.7447e-02, PNorm = 36.9371, GNorm = 0.6961, lr_0 = 7.0147e-04
Validation auc = 0.676056
Epoch 215
Validation auc = 0.682160
Epoch 216
Validation auc = 0.685915
Epoch 217
Validation auc = 0.683568
Epoch 218
Validation auc = 0.685915
Epoch 219
Loss = 1.3306e-02, PNorm = 36.9880, GNorm = 0.5257, lr_0 = 6.9548e-04
Validation auc = 0.687324
Epoch 220
Validation auc = 0.686385
Epoch 221
Validation auc = 0.671831
Epoch 222
Validation auc = 0.670892
Epoch 223
Validation auc = 0.666197
Epoch 224
Loss = 3.3123e-02, PNorm = 37.0302, GNorm = 0.6587, lr_0 = 6.8953e-04
Validation auc = 0.675117
Epoch 225
Validation auc = 0.680282
Epoch 226
Validation auc = 0.675587
Epoch 227
Validation auc = 0.682160
Epoch 228
Validation auc = 0.702817
Epoch 229
Loss = 3.0889e-02, PNorm = 37.0751, GNorm = 1.5512, lr_0 = 6.8364e-04
Validation auc = 0.690141
Epoch 230
Validation auc = 0.681690
Epoch 231
Validation auc = 0.681221
Epoch 232
Validation auc = 0.683099
Epoch 233
Validation auc = 0.684977
Epoch 234
Loss = 1.2250e-02, PNorm = 37.1153, GNorm = 0.6568, lr_0 = 6.7779e-04
Validation auc = 0.680282
Epoch 235
Validation auc = 0.679343
Epoch 236
Validation auc = 0.679343
Epoch 237
Validation auc = 0.688263
Epoch 238
Validation auc = 0.693897
Epoch 239
Loss = 2.1174e-02, PNorm = 37.1499, GNorm = 0.3470, lr_0 = 6.7200e-04
Validation auc = 0.697653
Epoch 240
Validation auc = 0.694836
Epoch 241
Validation auc = 0.694366
Epoch 242
Validation auc = 0.700000
Epoch 243
Validation auc = 0.692958
Epoch 244
Loss = 9.2838e-03, PNorm = 37.1814, GNorm = 0.1880, lr_0 = 6.6625e-04
Validation auc = 0.679812
Epoch 245
Validation auc = 0.675587
Epoch 246
Validation auc = 0.679343
Epoch 247
Validation auc = 0.681690
Epoch 248
Validation auc = 0.672770
Epoch 249
Loss = 2.8882e-02, PNorm = 37.2161, GNorm = 2.0214, lr_0 = 6.6056e-04
Validation auc = 0.683568
Epoch 250
Validation auc = 0.690610
Epoch 251
Validation auc = 0.686385
Epoch 252
Validation auc = 0.686385
Epoch 253
Validation auc = 0.696714
Epoch 254
Loss = 1.9512e-02, PNorm = 37.2536, GNorm = 0.4325, lr_0 = 6.5491e-04
Validation auc = 0.692488
Epoch 255
Validation auc = 0.681221
Epoch 256
Validation auc = 0.673709
Epoch 257
Validation auc = 0.665728
Epoch 258
Validation auc = 0.674178
Epoch 259
Loss = 2.6833e-02, PNorm = 37.2882, GNorm = 1.8223, lr_0 = 6.4931e-04
Validation auc = 0.668075
Epoch 260
Validation auc = 0.662441
Epoch 261
Validation auc = 0.666667
Epoch 262
Validation auc = 0.669484
Epoch 263
Validation auc = 0.675587
Epoch 264
Loss = 1.3195e-02, PNorm = 37.3259, GNorm = 0.1863, lr_0 = 6.4376e-04
Validation auc = 0.677465
Epoch 265
Validation auc = 0.672770
Epoch 266
Validation auc = 0.675117
Epoch 267
Validation auc = 0.681221
Epoch 268
Validation auc = 0.673709
Epoch 269
Loss = 5.2689e-03, PNorm = 37.3560, GNorm = 0.1960, lr_0 = 6.3826e-04
Validation auc = 0.669953
Epoch 270
Validation auc = 0.667136
Epoch 271
Validation auc = 0.665258
Epoch 272
Validation auc = 0.668075
Epoch 273
Validation auc = 0.681221
Epoch 274
Loss = 9.9363e-03, PNorm = 37.3799, GNorm = 0.6379, lr_0 = 6.3280e-04
Validation auc = 0.675117
Epoch 275
Validation auc = 0.671362
Epoch 276
Validation auc = 0.676526
Epoch 277
Validation auc = 0.680751
Epoch 278
Validation auc = 0.688263
Epoch 279
Loss = 1.0938e-02, PNorm = 37.4057, GNorm = 0.1499, lr_0 = 6.2739e-04
Validation auc = 0.680282
Epoch 280
Validation auc = 0.674178
Epoch 281
Validation auc = 0.673239
Epoch 282
Validation auc = 0.676526
Epoch 283
Validation auc = 0.685446
Epoch 284
Loss = 1.2384e-02, PNorm = 37.4296, GNorm = 0.6082, lr_0 = 6.2203e-04
Validation auc = 0.680282
Epoch 285
Validation auc = 0.669014
Epoch 286
Validation auc = 0.672770
Epoch 287
Validation auc = 0.690141
Epoch 288
Validation auc = 0.662441
Epoch 289
Loss = 2.4702e-02, PNorm = 37.4575, GNorm = 2.7311, lr_0 = 6.1671e-04
Validation auc = 0.657277
Epoch 290
Validation auc = 0.684507
Epoch 291
Validation auc = 0.682629
Epoch 292
Validation auc = 0.663850
Epoch 293
Validation auc = 0.668075
Epoch 294
Loss = 8.2062e-03, PNorm = 37.4963, GNorm = 0.3886, lr_0 = 6.1144e-04
Validation auc = 0.679343
Epoch 295
Validation auc = 0.675587
Epoch 296
Validation auc = 0.665728
Epoch 297
Validation auc = 0.665258
Epoch 298
Validation auc = 0.682160
Epoch 299
Loss = 1.1390e-02, PNorm = 37.5351, GNorm = 0.2296, lr_0 = 6.0621e-04
Validation auc = 0.697653
Model 0 best validation auc = 0.723944 on epoch 40
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.904275
Ensemble test auc = 0.904275
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=400, bias=False)
        (W_h): Linear(in_features=400, out_features=400, bias=False)
        (W_o): Linear(in_features=533, out_features=400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=628, out_features=1, bias=True)
  )
)
Number of parameters = 433,029
Moving model to cuda
Epoch 0
Validation auc = 0.396714
Epoch 1
Validation auc = 0.416901
Epoch 2
Validation auc = 0.445540
Epoch 3
Validation auc = 0.460563
Epoch 4
Loss = 7.0661e-01, PNorm = 32.8316, GNorm = 1.5567, lr_0 = 6.5000e-04
Validation auc = 0.472770
Epoch 5
Validation auc = 0.510329
Epoch 6
Validation auc = 0.577934
Epoch 7
Validation auc = 0.634272
Epoch 8
Validation auc = 0.660563
Epoch 9
Loss = 5.6025e-01, PNorm = 32.9225, GNorm = 1.9802, lr_0 = 9.9743e-04
Validation auc = 0.627230
Epoch 10
Validation auc = 0.653052
Epoch 11
Validation auc = 0.617840
Epoch 12
Validation auc = 0.660563
Epoch 13
Validation auc = 0.686385
Epoch 14
Loss = 5.2877e-01, PNorm = 32.9914, GNorm = 1.0834, lr_0 = 9.8890e-04
Validation auc = 0.658685
Epoch 15
Validation auc = 0.653991
Epoch 16
Validation auc = 0.661972
Epoch 17
Validation auc = 0.651174
Epoch 18
Validation auc = 0.635681
Epoch 19
Loss = 5.0955e-01, PNorm = 33.0595, GNorm = 0.7665, lr_0 = 9.8045e-04
Validation auc = 0.626761
Epoch 20
Validation auc = 0.617840
Epoch 21
Validation auc = 0.626761
Epoch 22
Validation auc = 0.645540
Epoch 23
Validation auc = 0.648826
Epoch 24
Loss = 4.8928e-01, PNorm = 33.1474, GNorm = 1.3216, lr_0 = 9.7207e-04
Validation auc = 0.662441
Epoch 25
Validation auc = 0.687324
Epoch 26
Validation auc = 0.716432
Epoch 27
Validation auc = 0.707981
Epoch 28
Validation auc = 0.680282
Epoch 29
Loss = 4.2532e-01, PNorm = 33.2474, GNorm = 1.2340, lr_0 = 9.6376e-04
Validation auc = 0.658685
Epoch 30
Validation auc = 0.651174
Epoch 31
Validation auc = 0.675117
Epoch 32
Validation auc = 0.682160
Epoch 33
Validation auc = 0.695305
Epoch 34
Loss = 4.2666e-01, PNorm = 33.3628, GNorm = 2.2883, lr_0 = 9.5552e-04
Validation auc = 0.741315
Epoch 35
Validation auc = 0.678873
Epoch 36
Validation auc = 0.669953
Epoch 37
Validation auc = 0.694366
Epoch 38
Validation auc = 0.696244
Epoch 39
Loss = 4.2240e-01, PNorm = 33.4690, GNorm = 1.1836, lr_0 = 9.4735e-04
Validation auc = 0.682160
Epoch 40
Validation auc = 0.669484
Epoch 41
Validation auc = 0.683568
Epoch 42
Validation auc = 0.679812
Epoch 43
Validation auc = 0.663380
Epoch 44
Loss = 3.4445e-01, PNorm = 33.5945, GNorm = 0.6213, lr_0 = 9.3925e-04
Validation auc = 0.677934
Epoch 45
Validation auc = 0.694836
Epoch 46
Validation auc = 0.696244
Epoch 47
Validation auc = 0.730047
Epoch 48
Validation auc = 0.723474
Epoch 49
Loss = 3.4276e-01, PNorm = 33.7243, GNorm = 4.1135, lr_0 = 9.3122e-04
Validation auc = 0.674648
Epoch 50
Validation auc = 0.691080
Epoch 51
Validation auc = 0.665728
Epoch 52
Validation auc = 0.655399
Epoch 53
Validation auc = 0.680751
Epoch 54
Loss = 3.2197e-01, PNorm = 33.8528, GNorm = 0.9573, lr_0 = 9.2326e-04
Validation auc = 0.695775
Epoch 55
Validation auc = 0.689671
Epoch 56
Validation auc = 0.692958
Epoch 57
Validation auc = 0.724883
Epoch 58
Validation auc = 0.720188
Epoch 59
Loss = 2.8475e-01, PNorm = 33.9974, GNorm = 3.0579, lr_0 = 9.1537e-04
Validation auc = 0.698592
Epoch 60
Validation auc = 0.712207
Epoch 61
Validation auc = 0.684507
Epoch 62
Validation auc = 0.688732
Epoch 63
Validation auc = 0.715493
Epoch 64
Loss = 2.5658e-01, PNorm = 34.1506, GNorm = 0.9266, lr_0 = 9.0754e-04
Validation auc = 0.691080
Epoch 65
Validation auc = 0.693897
Epoch 66
Validation auc = 0.689202
Epoch 67
Validation auc = 0.689671
Epoch 68
Validation auc = 0.689671
Epoch 69
Loss = 2.4509e-01, PNorm = 34.3048, GNorm = 3.5575, lr_0 = 8.9978e-04
Validation auc = 0.686385
Epoch 70
Validation auc = 0.708920
Epoch 71
Validation auc = 0.677465
Epoch 72
Validation auc = 0.715023
Epoch 73
Validation auc = 0.689202
Epoch 74
Loss = 2.1590e-01, PNorm = 34.4530, GNorm = 0.5277, lr_0 = 8.9209e-04
Validation auc = 0.702347
Epoch 75
Validation auc = 0.707512
Epoch 76
Validation auc = 0.698122
Epoch 77
Validation auc = 0.691549
Epoch 78
Validation auc = 0.676526
Epoch 79
Loss = 1.7320e-01, PNorm = 34.5914, GNorm = 0.7549, lr_0 = 8.8447e-04
Validation auc = 0.703756
Epoch 80
Validation auc = 0.688263
Epoch 81
Validation auc = 0.688263
Epoch 82
Validation auc = 0.713615
Epoch 83
Validation auc = 0.659624
Epoch 84
Loss = 2.4098e-01, PNorm = 34.7311, GNorm = 3.4689, lr_0 = 8.7691e-04
Validation auc = 0.688732
Epoch 85
Validation auc = 0.684977
Epoch 86
Validation auc = 0.662441
Epoch 87
Validation auc = 0.687793
Epoch 88
Validation auc = 0.708451
Epoch 89
Loss = 2.0961e-01, PNorm = 34.8564, GNorm = 1.5456, lr_0 = 8.6941e-04
Validation auc = 0.683099
Epoch 90
Validation auc = 0.663380
Epoch 91
Validation auc = 0.698592
Epoch 92
Validation auc = 0.692488
Epoch 93
Validation auc = 0.659624
Epoch 94
Loss = 2.0957e-01, PNorm = 34.9847, GNorm = 2.7578, lr_0 = 8.6198e-04
Validation auc = 0.659624
Epoch 95
Validation auc = 0.681690
Epoch 96
Validation auc = 0.674178
Epoch 97
Validation auc = 0.668545
Epoch 98
Validation auc = 0.678873
Epoch 99
Loss = 1.4067e-01, PNorm = 35.1201, GNorm = 1.0331, lr_0 = 8.5461e-04
Validation auc = 0.707512
Epoch 100
Validation auc = 0.693897
Epoch 101
Validation auc = 0.674648
Epoch 102
Validation auc = 0.678404
Epoch 103
Validation auc = 0.697183
Epoch 104
Loss = 1.9724e-01, PNorm = 35.2350, GNorm = 2.1938, lr_0 = 8.4730e-04
Validation auc = 0.679812
Epoch 105
Validation auc = 0.663850
Epoch 106
Validation auc = 0.666667
Epoch 107
Validation auc = 0.687324
Epoch 108
Validation auc = 0.682629
Epoch 109
Loss = 1.1263e-01, PNorm = 35.3484, GNorm = 2.4854, lr_0 = 8.4006e-04
Validation auc = 0.686385
Epoch 110
Validation auc = 0.712207
Epoch 111
Validation auc = 0.688263
Epoch 112
Validation auc = 0.671831
Epoch 113
Validation auc = 0.691080
Epoch 114
Loss = 1.0877e-01, PNorm = 35.4531, GNorm = 1.6951, lr_0 = 8.3288e-04
Validation auc = 0.674648
Epoch 115
Validation auc = 0.655399
Epoch 116
Validation auc = 0.686385
Epoch 117
Validation auc = 0.665728
Epoch 118
Validation auc = 0.648357
Epoch 119
Loss = 1.7769e-01, PNorm = 35.5660, GNorm = 3.6944, lr_0 = 8.2576e-04
Validation auc = 0.700939
Epoch 120
Validation auc = 0.712676
Epoch 121
Validation auc = 0.666667
Epoch 122
Validation auc = 0.662911
Epoch 123
Validation auc = 0.669484
Epoch 124
Loss = 1.3085e-01, PNorm = 35.6701, GNorm = 1.3978, lr_0 = 8.1870e-04
Validation auc = 0.674648
Epoch 125
Validation auc = 0.665258
Epoch 126
Validation auc = 0.664789
Epoch 127
Validation auc = 0.689671
Epoch 128
Validation auc = 0.703756
Epoch 129
Loss = 1.1367e-01, PNorm = 35.7892, GNorm = 0.7555, lr_0 = 8.1170e-04
Validation auc = 0.688263
Epoch 130
Validation auc = 0.673709
Epoch 131
Validation auc = 0.686385
Epoch 132
Validation auc = 0.697653
Epoch 133
Validation auc = 0.696714
Epoch 134
Loss = 8.3605e-02, PNorm = 35.8921, GNorm = 0.4032, lr_0 = 8.0476e-04
Validation auc = 0.686854
Epoch 135
Validation auc = 0.689671
Epoch 136
Validation auc = 0.688732
Epoch 137
Validation auc = 0.680282
Epoch 138
Validation auc = 0.650235
Epoch 139
Loss = 4.2065e-02, PNorm = 35.9811, GNorm = 0.3075, lr_0 = 7.9788e-04
Validation auc = 0.646948
Epoch 140
Validation auc = 0.684977
Epoch 141
Validation auc = 0.689671
Epoch 142
Validation auc = 0.679812
Epoch 143
Validation auc = 0.685446
Epoch 144
Loss = 1.3206e-01, PNorm = 36.0656, GNorm = 1.6075, lr_0 = 7.9106e-04
Validation auc = 0.711737
Epoch 145
Validation auc = 0.690610
Epoch 146
Validation auc = 0.661972
Epoch 147
Validation auc = 0.669484
Epoch 148
Validation auc = 0.690141
Epoch 149
Loss = 1.1144e-01, PNorm = 36.1459, GNorm = 1.1305, lr_0 = 7.8430e-04
Validation auc = 0.682160
Epoch 150
Validation auc = 0.684038
Epoch 151
Validation auc = 0.682160
Epoch 152
Validation auc = 0.692488
Epoch 153
Validation auc = 0.684977
Epoch 154
Loss = 5.8080e-02, PNorm = 36.2263, GNorm = 1.0802, lr_0 = 7.7759e-04
Validation auc = 0.663380
Epoch 155
Validation auc = 0.664789
Epoch 156
Validation auc = 0.680282
Epoch 157
Validation auc = 0.677934
Epoch 158
Validation auc = 0.668075
Epoch 159
Loss = 7.7012e-02, PNorm = 36.3053, GNorm = 0.4852, lr_0 = 7.7095e-04
Validation auc = 0.674178
Epoch 160
Validation auc = 0.682160
Epoch 161
Validation auc = 0.688732
Epoch 162
Validation auc = 0.696244
Epoch 163
Validation auc = 0.684507
Epoch 164
Loss = 8.6457e-02, PNorm = 36.3838, GNorm = 2.0923, lr_0 = 7.6436e-04
Validation auc = 0.685446
Epoch 165
Validation auc = 0.685446
Epoch 166
Validation auc = 0.685915
Epoch 167
Validation auc = 0.676526
Epoch 168
Validation auc = 0.686385
Epoch 169
Loss = 9.1361e-02, PNorm = 36.4586, GNorm = 4.1907, lr_0 = 7.5782e-04
Validation auc = 0.684977
Epoch 170
Validation auc = 0.676526
Epoch 171
Validation auc = 0.676995
Epoch 172
Validation auc = 0.683099
Epoch 173
Validation auc = 0.674178
Epoch 174
Loss = 5.4809e-02, PNorm = 36.5304, GNorm = 1.3886, lr_0 = 7.5134e-04
Validation auc = 0.669484
Epoch 175
Validation auc = 0.688732
Epoch 176
Validation auc = 0.688732
Epoch 177
Validation auc = 0.671831
Epoch 178
Validation auc = 0.676995
Epoch 179
Loss = 5.0868e-02, PNorm = 36.6089, GNorm = 0.5981, lr_0 = 7.4492e-04
Validation auc = 0.677465
Epoch 180
Validation auc = 0.668545
Epoch 181
Validation auc = 0.676995
Epoch 182
Validation auc = 0.691549
Epoch 183
Validation auc = 0.692019
Epoch 184
Loss = 4.7127e-02, PNorm = 36.6880, GNorm = 0.7004, lr_0 = 7.3855e-04
Validation auc = 0.677934
Epoch 185
Validation auc = 0.666667
Epoch 186
Validation auc = 0.662911
Epoch 187
Validation auc = 0.663850
Epoch 188
Validation auc = 0.661502
Epoch 189
Loss = 8.0952e-02, PNorm = 36.7531, GNorm = 3.0350, lr_0 = 7.3224e-04
Validation auc = 0.671362
Epoch 190
Validation auc = 0.686385
Epoch 191
Validation auc = 0.679812
Epoch 192
Validation auc = 0.675587
Epoch 193
Validation auc = 0.680282
Epoch 194
Loss = 4.2435e-02, PNorm = 36.8207, GNorm = 1.3093, lr_0 = 7.2598e-04
Validation auc = 0.686385
Epoch 195
Validation auc = 0.683568
Epoch 196
Validation auc = 0.675117
Epoch 197
Validation auc = 0.678873
Epoch 198
Validation auc = 0.690141
Epoch 199
Loss = 3.3604e-02, PNorm = 36.8838, GNorm = 1.2920, lr_0 = 7.1977e-04
Validation auc = 0.684507
Epoch 200
Validation auc = 0.685915
Epoch 201
Validation auc = 0.683568
Epoch 202
Validation auc = 0.666197
Epoch 203
Validation auc = 0.660563
Epoch 204
Loss = 1.9027e-02, PNorm = 36.9473, GNorm = 0.4581, lr_0 = 7.1362e-04
Validation auc = 0.657746
Epoch 205
Validation auc = 0.674178
Epoch 206
Validation auc = 0.685446
Epoch 207
Validation auc = 0.687793
Epoch 208
Validation auc = 0.693897
Epoch 209
Loss = 2.7533e-02, PNorm = 37.0137, GNorm = 0.6050, lr_0 = 7.0752e-04
Validation auc = 0.703756
Epoch 210
Validation auc = 0.696244
Epoch 211
Validation auc = 0.690610
Epoch 212
Validation auc = 0.695305
Epoch 213
Validation auc = 0.688263
Epoch 214
Loss = 2.0749e-02, PNorm = 37.0649, GNorm = 0.6121, lr_0 = 7.0147e-04
Validation auc = 0.671831
Epoch 215
Validation auc = 0.683099
Epoch 216
Validation auc = 0.684038
Epoch 217
Validation auc = 0.663850
Epoch 218
Validation auc = 0.649765
Epoch 219
Loss = 8.8628e-02, PNorm = 37.1229, GNorm = 1.2074, lr_0 = 6.9548e-04
Validation auc = 0.664319
Epoch 220
Validation auc = 0.671362
Epoch 221
Validation auc = 0.669953
Epoch 222
Validation auc = 0.675587
Epoch 223
Validation auc = 0.686385
Epoch 224
Loss = 6.5716e-02, PNorm = 37.1921, GNorm = 3.3505, lr_0 = 6.8953e-04
Validation auc = 0.692488
Epoch 225
Validation auc = 0.669484
Epoch 226
Validation auc = 0.666197
Epoch 227
Validation auc = 0.678873
Epoch 228
Validation auc = 0.684977
Epoch 229
Loss = 4.1458e-02, PNorm = 37.2653, GNorm = 0.8301, lr_0 = 6.8364e-04
Validation auc = 0.672770
Epoch 230
Validation auc = 0.658685
Epoch 231
Validation auc = 0.653991
Epoch 232
Validation auc = 0.661972
Epoch 233
Validation auc = 0.679812
Epoch 234
Loss = 3.3044e-02, PNorm = 37.3259, GNorm = 2.1415, lr_0 = 6.7779e-04
Validation auc = 0.672300
Epoch 235
Validation auc = 0.661972
Epoch 236
Validation auc = 0.676056
Epoch 237
Validation auc = 0.672770
Epoch 238
Validation auc = 0.659155
Epoch 239
Loss = 5.0393e-02, PNorm = 37.3823, GNorm = 1.3675, lr_0 = 6.7200e-04
Validation auc = 0.669014
Epoch 240
Validation auc = 0.684977
Epoch 241
Validation auc = 0.676526
Epoch 242
Validation auc = 0.675587
Epoch 243
Validation auc = 0.693897
Epoch 244
Loss = 2.1086e-02, PNorm = 37.4472, GNorm = 1.4951, lr_0 = 6.6625e-04
Validation auc = 0.695775
Epoch 245
Validation auc = 0.685915
Epoch 246
Validation auc = 0.673239
Epoch 247
Validation auc = 0.679812
Epoch 248
Validation auc = 0.685446
Epoch 249
Loss = 1.7302e-02, PNorm = 37.4972, GNorm = 0.3682, lr_0 = 6.6056e-04
Validation auc = 0.671831
Epoch 250
Validation auc = 0.665728
Epoch 251
Validation auc = 0.668075
Epoch 252
Validation auc = 0.678404
Epoch 253
Validation auc = 0.680282
Epoch 254
Loss = 1.1601e-02, PNorm = 37.5383, GNorm = 0.2669, lr_0 = 6.5491e-04
Validation auc = 0.672300
Epoch 255
Validation auc = 0.675117
Epoch 256
Validation auc = 0.674178
Epoch 257
Validation auc = 0.673709
Epoch 258
Validation auc = 0.674178
Epoch 259
Loss = 7.4275e-03, PNorm = 37.5742, GNorm = 0.2892, lr_0 = 6.4931e-04
Validation auc = 0.676056
Epoch 260
Validation auc = 0.680282
Epoch 261
Validation auc = 0.698592
Epoch 262
Validation auc = 0.700000
Epoch 263
Validation auc = 0.691549
Epoch 264
Loss = 1.6430e-02, PNorm = 37.6127, GNorm = 0.0857, lr_0 = 6.4376e-04
Validation auc = 0.691549
Epoch 265
Validation auc = 0.691080
Epoch 266
Validation auc = 0.685446
Epoch 267
Validation auc = 0.678404
Epoch 268
Validation auc = 0.670423
Epoch 269
Loss = 6.1711e-03, PNorm = 37.6479, GNorm = 0.1295, lr_0 = 6.3826e-04
Validation auc = 0.667606
Epoch 270
Validation auc = 0.670892
Epoch 271
Validation auc = 0.674648
Epoch 272
Validation auc = 0.676995
Epoch 273
Validation auc = 0.680751
Epoch 274
Loss = 7.1689e-03, PNorm = 37.6827, GNorm = 0.1887, lr_0 = 6.3280e-04
Validation auc = 0.686854
Epoch 275
Validation auc = 0.693427
Epoch 276
Validation auc = 0.689671
Epoch 277
Validation auc = 0.681690
Epoch 278
Validation auc = 0.677465
Epoch 279
Loss = 7.0749e-03, PNorm = 37.7129, GNorm = 0.2725, lr_0 = 6.2739e-04
Validation auc = 0.673239
Epoch 280
Validation auc = 0.672300
Epoch 281
Validation auc = 0.671831
Epoch 282
Validation auc = 0.664789
Epoch 283
Validation auc = 0.670423
Epoch 284
Loss = 5.2959e-03, PNorm = 37.7405, GNorm = 0.1797, lr_0 = 6.2203e-04
Validation auc = 0.674648
Epoch 285
Validation auc = 0.680751
Epoch 286
Validation auc = 0.681690
Epoch 287
Validation auc = 0.679343
Epoch 288
Validation auc = 0.684977
Epoch 289
Loss = 5.8613e-03, PNorm = 37.7716, GNorm = 0.3591, lr_0 = 6.1671e-04
Validation auc = 0.688732
Epoch 290
Validation auc = 0.686854
Epoch 291
Validation auc = 0.683099
Epoch 292
Validation auc = 0.684977
Epoch 293
Validation auc = 0.690610
Epoch 294
Loss = 8.5331e-03, PNorm = 37.8024, GNorm = 0.3695, lr_0 = 6.1144e-04
Validation auc = 0.685446
Epoch 295
Validation auc = 0.681690
Epoch 296
Validation auc = 0.673239
Epoch 297
Validation auc = 0.669953
Epoch 298
Validation auc = 0.673239
Epoch 299
Loss = 7.8544e-03, PNorm = 37.8315, GNorm = 0.4610, lr_0 = 6.0621e-04
Validation auc = 0.684977
Model 0 best validation auc = 0.741315 on epoch 34
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.880795
Ensemble test auc = 0.880795
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=400, bias=False)
        (W_h): Linear(in_features=400, out_features=400, bias=False)
        (W_o): Linear(in_features=533, out_features=400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=628, out_features=1, bias=True)
  )
)
Number of parameters = 433,029
Moving model to cuda
Epoch 0
Validation auc = 0.393897
Epoch 1
Validation auc = 0.415493
Epoch 2
Validation auc = 0.438967
Epoch 3
Validation auc = 0.452582
Epoch 4
Loss = 7.6307e-01, PNorm = 32.8342, GNorm = 2.0532, lr_0 = 6.5000e-04
Validation auc = 0.475587
Epoch 5
Validation auc = 0.513615
Epoch 6
Validation auc = 0.549296
Epoch 7
Validation auc = 0.602817
Epoch 8
Validation auc = 0.624883
Epoch 9
Loss = 6.5052e-01, PNorm = 32.9185, GNorm = 3.5473, lr_0 = 9.9743e-04
Validation auc = 0.611737
Epoch 10
Validation auc = 0.612207
Epoch 11
Validation auc = 0.640376
Epoch 12
Validation auc = 0.630986
Epoch 13
Validation auc = 0.681690
Epoch 14
Loss = 5.1414e-01, PNorm = 32.9973, GNorm = 1.8542, lr_0 = 9.8890e-04
Validation auc = 0.652113
Epoch 15
Validation auc = 0.651643
Epoch 16
Validation auc = 0.643192
Epoch 17
Validation auc = 0.620657
Epoch 18
Validation auc = 0.614085
Epoch 19
Loss = 4.6494e-01, PNorm = 33.0765, GNorm = 1.3773, lr_0 = 9.8045e-04
Validation auc = 0.619718
Epoch 20
Validation auc = 0.650704
Epoch 21
Validation auc = 0.648826
Epoch 22
Validation auc = 0.727700
Epoch 23
Validation auc = 0.671831
Epoch 24
Loss = 4.4655e-01, PNorm = 33.1657, GNorm = 1.3309, lr_0 = 9.7207e-04
Validation auc = 0.686385
Epoch 25
Validation auc = 0.708451
Epoch 26
Validation auc = 0.658685
Epoch 27
Validation auc = 0.646948
Epoch 28
Validation auc = 0.677934
Epoch 29
Loss = 5.0266e-01, PNorm = 33.2667, GNorm = 3.2550, lr_0 = 9.6376e-04
Validation auc = 0.679812
Epoch 30
Validation auc = 0.663380
Epoch 31
Validation auc = 0.713615
Epoch 32
Validation auc = 0.710329
Epoch 33
Validation auc = 0.664789
Epoch 34
Loss = 4.2786e-01, PNorm = 33.3721, GNorm = 2.2756, lr_0 = 9.5552e-04
Validation auc = 0.694836
Epoch 35
Validation auc = 0.722066
Epoch 36
Validation auc = 0.697183
Epoch 37
Validation auc = 0.672770
Epoch 38
Validation auc = 0.711268
Epoch 39
Loss = 3.8267e-01, PNorm = 33.5034, GNorm = 1.5530, lr_0 = 9.4735e-04
Validation auc = 0.708920
Epoch 40
Validation auc = 0.684038
Epoch 41
Validation auc = 0.689671
Epoch 42
Validation auc = 0.715962
Epoch 43
Validation auc = 0.697653
Epoch 44
Loss = 3.9280e-01, PNorm = 33.6354, GNorm = 3.4139, lr_0 = 9.3925e-04
Validation auc = 0.713146
Epoch 45
Validation auc = 0.703756
Epoch 46
Validation auc = 0.676526
Epoch 47
Validation auc = 0.720657
Epoch 48
Validation auc = 0.682160
Epoch 49
Loss = 3.4200e-01, PNorm = 33.7762, GNorm = 4.5016, lr_0 = 9.3122e-04
Validation auc = 0.720188
Epoch 50
Validation auc = 0.716432
Epoch 51
Validation auc = 0.673709
Epoch 52
Validation auc = 0.698592
Epoch 53
Validation auc = 0.694836
Epoch 54
Loss = 2.7157e-01, PNorm = 33.9165, GNorm = 1.2033, lr_0 = 9.2326e-04
Validation auc = 0.681221
Epoch 55
Validation auc = 0.715493
Epoch 56
Validation auc = 0.724883
Epoch 57
Validation auc = 0.682160
Epoch 58
Validation auc = 0.709859
Epoch 59
Loss = 3.2620e-01, PNorm = 34.0537, GNorm = 2.8986, lr_0 = 9.1537e-04
Validation auc = 0.695775
Epoch 60
Validation auc = 0.690610
Epoch 61
Validation auc = 0.721596
Epoch 62
Validation auc = 0.718310
Epoch 63
Validation auc = 0.692019
Epoch 64
Loss = 2.3377e-01, PNorm = 34.1877, GNorm = 2.0580, lr_0 = 9.0754e-04
Validation auc = 0.710798
Epoch 65
Validation auc = 0.725822
Epoch 66
Validation auc = 0.697653
Epoch 67
Validation auc = 0.714554
Epoch 68
Validation auc = 0.709390
Epoch 69
Loss = 2.1166e-01, PNorm = 34.3296, GNorm = 3.2261, lr_0 = 8.9978e-04
Validation auc = 0.696714
Epoch 70
Validation auc = 0.724883
Epoch 71
Validation auc = 0.698592
Epoch 72
Validation auc = 0.679343
Epoch 73
Validation auc = 0.721127
Epoch 74
Loss = 2.0101e-01, PNorm = 34.4649, GNorm = 3.7003, lr_0 = 8.9209e-04
Validation auc = 0.704695
Epoch 75
Validation auc = 0.701408
Epoch 76
Validation auc = 0.728169
Epoch 77
Validation auc = 0.694366
Epoch 78
Validation auc = 0.688263
Epoch 79
Loss = 2.2654e-01, PNorm = 34.6005, GNorm = 1.0655, lr_0 = 8.8447e-04
Validation auc = 0.710798
Epoch 80
Validation auc = 0.696714
Epoch 81
Validation auc = 0.689671
Epoch 82
Validation auc = 0.707042
Epoch 83
Validation auc = 0.694366
Epoch 84
Loss = 2.3283e-01, PNorm = 34.7254, GNorm = 1.5109, lr_0 = 8.7691e-04
Validation auc = 0.682160
Epoch 85
Validation auc = 0.687324
Epoch 86
Validation auc = 0.684038
Epoch 87
Validation auc = 0.681221
Epoch 88
Validation auc = 0.711268
Epoch 89
Loss = 1.6050e-01, PNorm = 34.8534, GNorm = 0.9182, lr_0 = 8.6941e-04
Validation auc = 0.696244
Epoch 90
Validation auc = 0.699531
Epoch 91
Validation auc = 0.709859
Epoch 92
Validation auc = 0.712207
Epoch 93
Validation auc = 0.685446
Epoch 94
Loss = 1.1845e-01, PNorm = 34.9760, GNorm = 1.1799, lr_0 = 8.6198e-04
Validation auc = 0.698122
Epoch 95
Validation auc = 0.704695
Epoch 96
Validation auc = 0.692958
Epoch 97
Validation auc = 0.702347
Epoch 98
Validation auc = 0.699531
Epoch 99
Loss = 1.3642e-01, PNorm = 35.0874, GNorm = 2.5609, lr_0 = 8.5461e-04
Validation auc = 0.703286
Epoch 100
Validation auc = 0.680282
Epoch 101
Validation auc = 0.664789
Epoch 102
Validation auc = 0.682629
Epoch 103
Validation auc = 0.672770
Epoch 104
Loss = 1.8318e-01, PNorm = 35.1931, GNorm = 3.0191, lr_0 = 8.4730e-04
Validation auc = 0.685446
Epoch 105
Validation auc = 0.683099
Epoch 106
Validation auc = 0.651643
Epoch 107
Validation auc = 0.649765
Epoch 108
Validation auc = 0.658685
Epoch 109
Loss = 1.0916e-01, PNorm = 35.2930, GNorm = 0.8256, lr_0 = 8.4006e-04
Validation auc = 0.665728
Epoch 110
Validation auc = 0.682160
Epoch 111
Validation auc = 0.705634
Epoch 112
Validation auc = 0.696244
Epoch 113
Validation auc = 0.677465
Epoch 114
Loss = 8.1829e-02, PNorm = 35.3966, GNorm = 2.0920, lr_0 = 8.3288e-04
Validation auc = 0.688732
Epoch 115
Validation auc = 0.656808
Epoch 116
Validation auc = 0.673709
Epoch 117
Validation auc = 0.646479
Epoch 118
Validation auc = 0.607042
Epoch 119
Loss = 1.6880e-01, PNorm = 35.4842, GNorm = 1.8537, lr_0 = 8.2576e-04
Validation auc = 0.631455
Epoch 120
Validation auc = 0.650235
Epoch 121
Validation auc = 0.653521
Epoch 122
Validation auc = 0.661033
Epoch 123
Validation auc = 0.686385
Epoch 124
Loss = 1.1696e-01, PNorm = 35.5961, GNorm = 2.1393, lr_0 = 8.1870e-04
Validation auc = 0.692488
Epoch 125
Validation auc = 0.673239
Epoch 126
Validation auc = 0.673239
Epoch 127
Validation auc = 0.674648
Epoch 128
Validation auc = 0.669014
Epoch 129
Loss = 8.2616e-02, PNorm = 35.6952, GNorm = 0.6919, lr_0 = 8.1170e-04
Validation auc = 0.660094
Epoch 130
Validation auc = 0.676056
Epoch 131
Validation auc = 0.694366
Epoch 132
Validation auc = 0.677934
Epoch 133
Validation auc = 0.665258
Epoch 134
Loss = 2.2936e-02, PNorm = 35.7907, GNorm = 0.7171, lr_0 = 8.0476e-04
Validation auc = 0.657277
Epoch 135
Validation auc = 0.692958
Epoch 136
Validation auc = 0.695305
Epoch 137
Validation auc = 0.660563
Epoch 138
Validation auc = 0.679812
Epoch 139
Loss = 1.0144e-01, PNorm = 35.8746, GNorm = 1.9608, lr_0 = 7.9788e-04
Validation auc = 0.699531
Epoch 140
Validation auc = 0.667136
Epoch 141
Validation auc = 0.649296
Epoch 142
Validation auc = 0.651643
Epoch 143
Validation auc = 0.659624
Epoch 144
Loss = 1.0911e-01, PNorm = 35.9599, GNorm = 1.7777, lr_0 = 7.9106e-04
Validation auc = 0.654930
Epoch 145
Validation auc = 0.650235
Epoch 146
Validation auc = 0.657746
Epoch 147
Validation auc = 0.661972
Epoch 148
Validation auc = 0.656808
Epoch 149
Loss = 4.7162e-02, PNorm = 36.0452, GNorm = 0.3816, lr_0 = 7.8430e-04
Validation auc = 0.650704
Epoch 150
Validation auc = 0.647418
Epoch 151
Validation auc = 0.659155
Epoch 152
Validation auc = 0.665728
Epoch 153
Validation auc = 0.656808
Epoch 154
Loss = 5.9443e-02, PNorm = 36.1159, GNorm = 0.7600, lr_0 = 7.7759e-04
Validation auc = 0.665258
Epoch 155
Validation auc = 0.661502
Epoch 156
Validation auc = 0.654930
Epoch 157
Validation auc = 0.647418
Epoch 158
Validation auc = 0.665258
Epoch 159
Loss = 4.2370e-02, PNorm = 36.1787, GNorm = 1.1462, lr_0 = 7.7095e-04
Validation auc = 0.669953
Epoch 160
Validation auc = 0.666667
Epoch 161
Validation auc = 0.671362
Epoch 162
Validation auc = 0.684507
Epoch 163
Validation auc = 0.684977
Epoch 164
Loss = 8.1343e-02, PNorm = 36.2420, GNorm = 1.4458, lr_0 = 7.6436e-04
Validation auc = 0.682160
Epoch 165
Validation auc = 0.671362
Epoch 166
Validation auc = 0.671831
Epoch 167
Validation auc = 0.675587
Epoch 168
Validation auc = 0.653521
Epoch 169
Loss = 8.2340e-02, PNorm = 36.3068, GNorm = 1.9459, lr_0 = 7.5782e-04
Validation auc = 0.661502
Epoch 170
Validation auc = 0.657746
Epoch 171
Validation auc = 0.640376
Epoch 172
Validation auc = 0.671362
Epoch 173
Validation auc = 0.683568
Epoch 174
Loss = 7.4430e-02, PNorm = 36.3653, GNorm = 0.4140, lr_0 = 7.5134e-04
Validation auc = 0.657277
Epoch 175
Validation auc = 0.653521
Epoch 176
Validation auc = 0.675117
Epoch 177
Validation auc = 0.667606
Epoch 178
Validation auc = 0.647418
Epoch 179
Loss = 6.7905e-02, PNorm = 36.4375, GNorm = 0.4314, lr_0 = 7.4492e-04
Validation auc = 0.645540
Epoch 180
Validation auc = 0.651643
Epoch 181
Validation auc = 0.655869
Epoch 182
Validation auc = 0.660094
Epoch 183
Validation auc = 0.661972
Epoch 184
Loss = 8.2956e-02, PNorm = 36.5088, GNorm = 0.2985, lr_0 = 7.3855e-04
Validation auc = 0.675587
Epoch 185
Validation auc = 0.677465
Epoch 186
Validation auc = 0.664789
Epoch 187
Validation auc = 0.661972
Epoch 188
Validation auc = 0.660563
Epoch 189
Loss = 3.7358e-02, PNorm = 36.5748, GNorm = 0.5093, lr_0 = 7.3224e-04
Validation auc = 0.653991
Epoch 190
Validation auc = 0.647418
Epoch 191
Validation auc = 0.648357
Epoch 192
Validation auc = 0.666667
Epoch 193
Validation auc = 0.670423
Epoch 194
Loss = 1.8563e-02, PNorm = 36.6384, GNorm = 0.3644, lr_0 = 7.2598e-04
Validation auc = 0.667606
Epoch 195
Validation auc = 0.694836
Epoch 196
Validation auc = 0.669014
Epoch 197
Validation auc = 0.623944
Epoch 198
Validation auc = 0.652582
Epoch 199
Loss = 6.1202e-02, PNorm = 36.6981, GNorm = 3.3432, lr_0 = 7.1977e-04
Validation auc = 0.682629
Epoch 200
Validation auc = 0.649296
Epoch 201
Validation auc = 0.628638
Epoch 202
Validation auc = 0.640376
Epoch 203
Validation auc = 0.669014
Epoch 204
Loss = 8.1833e-02, PNorm = 36.7707, GNorm = 1.7306, lr_0 = 7.1362e-04
Validation auc = 0.660563
Epoch 205
Validation auc = 0.648826
Epoch 206
Validation auc = 0.675117
Epoch 207
Validation auc = 0.712207
Epoch 208
Validation auc = 0.706573
Epoch 209
Loss = 2.6157e-02, PNorm = 36.8419, GNorm = 0.4931, lr_0 = 7.0752e-04
Validation auc = 0.686854
Epoch 210
Validation auc = 0.672770
Epoch 211
Validation auc = 0.664319
Epoch 212
Validation auc = 0.672300
Epoch 213
Validation auc = 0.680751
Epoch 214
Loss = 5.2339e-02, PNorm = 36.8926, GNorm = 0.5158, lr_0 = 7.0147e-04
Validation auc = 0.675117
Epoch 215
Validation auc = 0.669484
Epoch 216
Validation auc = 0.676995
Epoch 217
Validation auc = 0.682160
Epoch 218
Validation auc = 0.682629
Epoch 219
Loss = 1.0506e-02, PNorm = 36.9440, GNorm = 0.2268, lr_0 = 6.9548e-04
Validation auc = 0.685446
Epoch 220
Validation auc = 0.700939
Epoch 221
Validation auc = 0.698122
Epoch 222
Validation auc = 0.679812
Epoch 223
Validation auc = 0.667606
Epoch 224
Loss = 3.4785e-02, PNorm = 36.9912, GNorm = 1.7130, lr_0 = 6.8953e-04
Validation auc = 0.664789
Epoch 225
Validation auc = 0.666667
Epoch 226
Validation auc = 0.657746
Epoch 227
Validation auc = 0.665728
Epoch 228
Validation auc = 0.670423
Epoch 229
Loss = 1.6462e-02, PNorm = 37.0369, GNorm = 0.5172, lr_0 = 6.8364e-04
Validation auc = 0.668075
Epoch 230
Validation auc = 0.673709
Epoch 231
Validation auc = 0.683099
Epoch 232
Validation auc = 0.685915
Epoch 233
Validation auc = 0.685446
Epoch 234
Loss = 1.5395e-02, PNorm = 37.0802, GNorm = 0.3928, lr_0 = 6.7779e-04
Validation auc = 0.677465
Epoch 235
Validation auc = 0.667606
Epoch 236
Validation auc = 0.672300
Epoch 237
Validation auc = 0.684507
Epoch 238
Validation auc = 0.665728
Epoch 239
Loss = 1.2619e-02, PNorm = 37.1172, GNorm = 0.4547, lr_0 = 6.7200e-04
Validation auc = 0.647418
Epoch 240
Validation auc = 0.650235
Epoch 241
Validation auc = 0.665728
Epoch 242
Validation auc = 0.675587
Epoch 243
Validation auc = 0.671831
Epoch 244
Loss = 4.2082e-02, PNorm = 37.1613, GNorm = 0.3942, lr_0 = 6.6625e-04
Validation auc = 0.670423
Epoch 245
Validation auc = 0.667606
Epoch 246
Validation auc = 0.664319
Epoch 247
Validation auc = 0.664319
Epoch 248
Validation auc = 0.665258
Epoch 249
Loss = 1.0548e-02, PNorm = 37.1958, GNorm = 0.1660, lr_0 = 6.6056e-04
Validation auc = 0.668545
Epoch 250
Validation auc = 0.669484
Epoch 251
Validation auc = 0.668075
Epoch 252
Validation auc = 0.665728
Epoch 253
Validation auc = 0.663850
Epoch 254
Loss = 7.6987e-03, PNorm = 37.2280, GNorm = 0.2595, lr_0 = 6.5491e-04
Validation auc = 0.659155
Epoch 255
Validation auc = 0.654460
Epoch 256
Validation auc = 0.653991
Epoch 257
Validation auc = 0.655399
Epoch 258
Validation auc = 0.658685
Epoch 259
Loss = 7.6788e-03, PNorm = 37.2525, GNorm = 0.3599, lr_0 = 6.4931e-04
Validation auc = 0.661502
Epoch 260
Validation auc = 0.662441
Epoch 261
Validation auc = 0.670423
Epoch 262
Validation auc = 0.670423
Epoch 263
Validation auc = 0.667136
Epoch 264
Loss = 3.3529e-03, PNorm = 37.2744, GNorm = 0.0862, lr_0 = 6.4376e-04
Validation auc = 0.661502
Epoch 265
Validation auc = 0.659155
Epoch 266
Validation auc = 0.665728
Epoch 267
Validation auc = 0.672300
Epoch 268
Validation auc = 0.669014
Epoch 269
Loss = 1.3694e-02, PNorm = 37.2982, GNorm = 0.4161, lr_0 = 6.3826e-04
Validation auc = 0.668075
Epoch 270
Validation auc = 0.670892
Epoch 271
Validation auc = 0.676526
Epoch 272
Validation auc = 0.678404
Epoch 273
Validation auc = 0.674178
Epoch 274
Loss = 3.8048e-03, PNorm = 37.3238, GNorm = 0.2778, lr_0 = 6.3280e-04
Validation auc = 0.673239
Epoch 275
Validation auc = 0.667606
Epoch 276
Validation auc = 0.671831
Epoch 277
Validation auc = 0.676995
Epoch 278
Validation auc = 0.671362
Epoch 279
Loss = 1.6710e-02, PNorm = 37.3454, GNorm = 0.9377, lr_0 = 6.2739e-04
Validation auc = 0.661033
Epoch 280
Validation auc = 0.656338
Epoch 281
Validation auc = 0.661033
Epoch 282
Validation auc = 0.666667
Epoch 283
Validation auc = 0.660563
Epoch 284
Loss = 2.1179e-02, PNorm = 37.3745, GNorm = 1.2358, lr_0 = 6.2203e-04
Validation auc = 0.665258
Epoch 285
Validation auc = 0.678404
Epoch 286
Validation auc = 0.681221
Epoch 287
Validation auc = 0.675117
Epoch 288
Validation auc = 0.675587
Epoch 289
Loss = 1.3686e-02, PNorm = 37.4079, GNorm = 0.0925, lr_0 = 6.1671e-04
Validation auc = 0.682629
Epoch 290
Validation auc = 0.685446
Epoch 291
Validation auc = 0.684507
Epoch 292
Validation auc = 0.677465
Epoch 293
Validation auc = 0.669484
Epoch 294
Loss = 1.2923e-02, PNorm = 37.4351, GNorm = 0.4729, lr_0 = 6.1144e-04
Validation auc = 0.669014
Epoch 295
Validation auc = 0.660563
Epoch 296
Validation auc = 0.652113
Epoch 297
Validation auc = 0.649296
Epoch 298
Validation auc = 0.656808
Epoch 299
Loss = 1.4713e-02, PNorm = 37.4645, GNorm = 1.1367, lr_0 = 6.0621e-04
Validation auc = 0.661502
Model 0 best validation auc = 0.728169 on epoch 76
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.877784
Ensemble test auc = 0.877784
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=400, bias=False)
        (W_h): Linear(in_features=400, out_features=400, bias=False)
        (W_o): Linear(in_features=533, out_features=400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=628, out_features=1, bias=True)
  )
)
Number of parameters = 433,029
Moving model to cuda
Epoch 0
Validation auc = 0.397183
Epoch 1
Validation auc = 0.407512
Epoch 2
Validation auc = 0.428638
Epoch 3
Validation auc = 0.453521
Epoch 4
Loss = 7.4578e-01, PNorm = 32.8281, GNorm = 1.6691, lr_0 = 6.5000e-04
Validation auc = 0.471831
Epoch 5
Validation auc = 0.498122
Epoch 6
Validation auc = 0.535681
Epoch 7
Validation auc = 0.584038
Epoch 8
Validation auc = 0.632394
Epoch 9
Loss = 6.2175e-01, PNorm = 32.9170, GNorm = 1.7347, lr_0 = 9.9743e-04
Validation auc = 0.640376
Epoch 10
Validation auc = 0.714085
Epoch 11
Validation auc = 0.637089
Epoch 12
Validation auc = 0.626291
Epoch 13
Validation auc = 0.635211
Epoch 14
Loss = 6.3305e-01, PNorm = 32.9823, GNorm = 2.5066, lr_0 = 9.8890e-04
Validation auc = 0.633333
Epoch 15
Validation auc = 0.620657
Epoch 16
Validation auc = 0.616432
Epoch 17
Validation auc = 0.623944
Epoch 18
Validation auc = 0.631455
Epoch 19
Loss = 5.0881e-01, PNorm = 33.0551, GNorm = 0.9341, lr_0 = 9.8045e-04
Validation auc = 0.639437
Epoch 20
Validation auc = 0.637559
Epoch 21
Validation auc = 0.630047
Epoch 22
Validation auc = 0.633803
Epoch 23
Validation auc = 0.655869
Epoch 24
Loss = 4.9132e-01, PNorm = 33.1503, GNorm = 2.3497, lr_0 = 9.7207e-04
Validation auc = 0.654930
Epoch 25
Validation auc = 0.631455
Epoch 26
Validation auc = 0.634742
Epoch 27
Validation auc = 0.650235
Epoch 28
Validation auc = 0.644601
Epoch 29
Loss = 4.6622e-01, PNorm = 33.2526, GNorm = 1.9069, lr_0 = 9.6376e-04
Validation auc = 0.656338
Epoch 30
Validation auc = 0.670892
Epoch 31
Validation auc = 0.655869
Epoch 32
Validation auc = 0.701878
Epoch 33
Validation auc = 0.657746
Epoch 34
Loss = 3.9571e-01, PNorm = 33.3658, GNorm = 0.7497, lr_0 = 9.5552e-04
Validation auc = 0.676526
Epoch 35
Validation auc = 0.674178
Epoch 36
Validation auc = 0.673709
Epoch 37
Validation auc = 0.695775
Epoch 38
Validation auc = 0.706103
Epoch 39
Loss = 3.8078e-01, PNorm = 33.4950, GNorm = 2.4416, lr_0 = 9.4735e-04
Validation auc = 0.717371
Epoch 40
Validation auc = 0.677934
Epoch 41
Validation auc = 0.687793
Epoch 42
Validation auc = 0.655869
Epoch 43
Validation auc = 0.642254
Epoch 44
Loss = 3.0720e-01, PNorm = 33.6355, GNorm = 1.6834, lr_0 = 9.3925e-04
Validation auc = 0.652113
Epoch 45
Validation auc = 0.688732
Epoch 46
Validation auc = 0.668545
Epoch 47
Validation auc = 0.691080
Epoch 48
Validation auc = 0.650235
Epoch 49
Loss = 3.1762e-01, PNorm = 33.8033, GNorm = 0.8721, lr_0 = 9.3122e-04
Validation auc = 0.689202
Epoch 50
Validation auc = 0.643662
Epoch 51
Validation auc = 0.683568
Epoch 52
Validation auc = 0.648357
Epoch 53
Validation auc = 0.670892
Epoch 54
Loss = 2.7491e-01, PNorm = 33.9435, GNorm = 3.1257, lr_0 = 9.2326e-04
Validation auc = 0.664319
Epoch 55
Validation auc = 0.656338
Epoch 56
Validation auc = 0.692958
Epoch 57
Validation auc = 0.665258
Epoch 58
Validation auc = 0.682160
Epoch 59
Loss = 2.7897e-01, PNorm = 34.0917, GNorm = 1.3025, lr_0 = 9.1537e-04
Validation auc = 0.695775
Epoch 60
Validation auc = 0.664789
Epoch 61
Validation auc = 0.711737
Epoch 62
Validation auc = 0.670892
Epoch 63
Validation auc = 0.684507
Epoch 64
Loss = 2.3386e-01, PNorm = 34.2342, GNorm = 4.6151, lr_0 = 9.0754e-04
Validation auc = 0.655869
Epoch 65
Validation auc = 0.679343
Epoch 66
Validation auc = 0.716901
Epoch 67
Validation auc = 0.674648
Epoch 68
Validation auc = 0.666197
Epoch 69
Loss = 2.5447e-01, PNorm = 34.3747, GNorm = 1.9369, lr_0 = 8.9978e-04
Validation auc = 0.684977
Epoch 70
Validation auc = 0.659624
Epoch 71
Validation auc = 0.678404
Epoch 72
Validation auc = 0.716901
Epoch 73
Validation auc = 0.668075
Epoch 74
Loss = 1.9211e-01, PNorm = 34.5053, GNorm = 3.6891, lr_0 = 8.9209e-04
Validation auc = 0.654460
Epoch 75
Validation auc = 0.689202
Epoch 76
Validation auc = 0.673239
Epoch 77
Validation auc = 0.693427
Epoch 78
Validation auc = 0.710798
Epoch 79
Loss = 1.9134e-01, PNorm = 34.6367, GNorm = 0.9602, lr_0 = 8.8447e-04
Validation auc = 0.671362
Epoch 80
Validation auc = 0.658685
Epoch 81
Validation auc = 0.670423
Epoch 82
Validation auc = 0.677465
Epoch 83
Validation auc = 0.691549
Epoch 84
Loss = 1.3385e-01, PNorm = 34.7738, GNorm = 1.9611, lr_0 = 8.7691e-04
Validation auc = 0.708451
Epoch 85
Validation auc = 0.688732
Epoch 86
Validation auc = 0.708920
Epoch 87
Validation auc = 0.684977
Epoch 88
Validation auc = 0.688263
Epoch 89
Loss = 1.5825e-01, PNorm = 34.8904, GNorm = 1.7801, lr_0 = 8.6941e-04
Validation auc = 0.710329
Epoch 90
Validation auc = 0.676526
Epoch 91
Validation auc = 0.699531
Epoch 92
Validation auc = 0.705634
Epoch 93
Validation auc = 0.656338
Epoch 94
Loss = 1.8419e-01, PNorm = 34.9993, GNorm = 4.1427, lr_0 = 8.6198e-04
Validation auc = 0.666667
Epoch 95
Validation auc = 0.691549
Epoch 96
Validation auc = 0.667606
Epoch 97
Validation auc = 0.662911
Epoch 98
Validation auc = 0.687324
Epoch 99
Loss = 1.4127e-01, PNorm = 35.1099, GNorm = 3.2543, lr_0 = 8.5461e-04
Validation auc = 0.660094
Epoch 100
Validation auc = 0.650704
Epoch 101
Validation auc = 0.677465
Epoch 102
Validation auc = 0.667606
Epoch 103
Validation auc = 0.658216
Epoch 104
Loss = 1.9581e-01, PNorm = 35.2193, GNorm = 1.2277, lr_0 = 8.4730e-04
Validation auc = 0.693427
Epoch 105
Validation auc = 0.664789
Epoch 106
Validation auc = 0.661502
Epoch 107
Validation auc = 0.682160
Epoch 108
Validation auc = 0.660094
Epoch 109
Loss = 8.0892e-02, PNorm = 35.3266, GNorm = 1.0559, lr_0 = 8.4006e-04
Validation auc = 0.667606
Epoch 110
Validation auc = 0.696244
Epoch 111
Validation auc = 0.698592
Epoch 112
Validation auc = 0.688263
Epoch 113
Validation auc = 0.686385
Epoch 114
Loss = 6.2274e-02, PNorm = 35.4379, GNorm = 0.9713, lr_0 = 8.3288e-04
Validation auc = 0.692488
Epoch 115
Validation auc = 0.689202
Epoch 116
Validation auc = 0.678873
Epoch 117
Validation auc = 0.680751
Epoch 118
Validation auc = 0.665258
Epoch 119
Loss = 5.3723e-02, PNorm = 35.5362, GNorm = 0.8024, lr_0 = 8.2576e-04
Validation auc = 0.674648
Epoch 120
Validation auc = 0.690610
Epoch 121
Validation auc = 0.688263
Epoch 122
Validation auc = 0.703286
Epoch 123
Validation auc = 0.676995
Epoch 124
Loss = 1.2275e-01, PNorm = 35.6204, GNorm = 1.9265, lr_0 = 8.1870e-04
Validation auc = 0.658216
Epoch 125
Validation auc = 0.662441
Epoch 126
Validation auc = 0.653991
Epoch 127
Validation auc = 0.684507
Epoch 128
Validation auc = 0.701408
Epoch 129
Loss = 1.0481e-01, PNorm = 35.7119, GNorm = 0.8985, lr_0 = 8.1170e-04
Validation auc = 0.683568
Epoch 130
Validation auc = 0.680751
Epoch 131
Validation auc = 0.682629
Epoch 132
Validation auc = 0.670892
Epoch 133
Validation auc = 0.684507
Epoch 134
Loss = 9.1533e-02, PNorm = 35.8073, GNorm = 1.2246, lr_0 = 8.0476e-04
Validation auc = 0.679343
Epoch 135
Validation auc = 0.668545
Epoch 136
Validation auc = 0.670423
Epoch 137
Validation auc = 0.671362
Epoch 138
Validation auc = 0.690610
Epoch 139
Loss = 7.0156e-02, PNorm = 35.8874, GNorm = 2.9699, lr_0 = 7.9788e-04
Validation auc = 0.688263
Epoch 140
Validation auc = 0.676995
Epoch 141
Validation auc = 0.671362
Epoch 142
Validation auc = 0.674178
Epoch 143
Validation auc = 0.684977
Epoch 144
Loss = 7.8608e-02, PNorm = 35.9599, GNorm = 2.8095, lr_0 = 7.9106e-04
Validation auc = 0.698122
Epoch 145
Validation auc = 0.675117
Epoch 146
Validation auc = 0.658685
Epoch 147
Validation auc = 0.672770
Epoch 148
Validation auc = 0.655869
Epoch 149
Loss = 7.0555e-02, PNorm = 36.0256, GNorm = 3.2720, lr_0 = 7.8430e-04
Validation auc = 0.649765
Epoch 150
Validation auc = 0.672300
Epoch 151
Validation auc = 0.661033
Epoch 152
Validation auc = 0.647418
Epoch 153
Validation auc = 0.660094
Epoch 154
Loss = 4.6509e-02, PNorm = 36.0992, GNorm = 1.6642, lr_0 = 7.7759e-04
Validation auc = 0.666197
Epoch 155
Validation auc = 0.668545
Epoch 156
Validation auc = 0.676526
Epoch 157
Validation auc = 0.684507
Epoch 158
Validation auc = 0.684977
Epoch 159
Loss = 6.8967e-02, PNorm = 36.1658, GNorm = 0.8284, lr_0 = 7.7095e-04
Validation auc = 0.682160
Epoch 160
Validation auc = 0.684038
Epoch 161
Validation auc = 0.672770
Epoch 162
Validation auc = 0.676056
Epoch 163
Validation auc = 0.673239
Epoch 164
Loss = 5.2160e-02, PNorm = 36.2290, GNorm = 1.5165, lr_0 = 7.6436e-04
Validation auc = 0.672300
Epoch 165
Validation auc = 0.680282
Epoch 166
Validation auc = 0.669014
Epoch 167
Validation auc = 0.676526
Epoch 168
Validation auc = 0.672770
Epoch 169
Loss = 4.0143e-02, PNorm = 36.2864, GNorm = 0.8446, lr_0 = 7.5782e-04
Validation auc = 0.634272
Epoch 170
Validation auc = 0.676995
Epoch 171
Validation auc = 0.683099
Epoch 172
Validation auc = 0.665258
Epoch 173
Validation auc = 0.670892
Epoch 174
Loss = 1.0475e-01, PNorm = 36.3655, GNorm = 4.0280, lr_0 = 7.5134e-04
Validation auc = 0.693897
Epoch 175
Validation auc = 0.685915
Epoch 176
Validation auc = 0.654930
Epoch 177
Validation auc = 0.670423
Epoch 178
Validation auc = 0.694366
Epoch 179
Loss = 9.8520e-02, PNorm = 36.4544, GNorm = 4.0341, lr_0 = 7.4492e-04
Validation auc = 0.666667
Epoch 180
Validation auc = 0.643192
Epoch 181
Validation auc = 0.651643
Epoch 182
Validation auc = 0.668075
Epoch 183
Validation auc = 0.670892
Epoch 184
Loss = 4.2883e-02, PNorm = 36.5282, GNorm = 1.7097, lr_0 = 7.3855e-04
Validation auc = 0.674178
Epoch 185
Validation auc = 0.688263
Epoch 186
Validation auc = 0.701878
Epoch 187
Validation auc = 0.710798
Epoch 188
Validation auc = 0.699531
Epoch 189
Loss = 2.2265e-02, PNorm = 36.5967, GNorm = 0.1639, lr_0 = 7.3224e-04
Validation auc = 0.687793
Epoch 190
Validation auc = 0.686385
Epoch 191
Validation auc = 0.700000
Epoch 192
Validation auc = 0.680751
Epoch 193
Validation auc = 0.672300
Epoch 194
Loss = 3.0699e-02, PNorm = 36.6514, GNorm = 0.2146, lr_0 = 7.2598e-04
Validation auc = 0.679343
Epoch 195
Validation auc = 0.694836
Epoch 196
Validation auc = 0.701878
Epoch 197
Validation auc = 0.687324
Epoch 198
Validation auc = 0.675587
Epoch 199
Loss = 1.7395e-02, PNorm = 36.6995, GNorm = 0.7009, lr_0 = 7.1977e-04
Validation auc = 0.676526
Epoch 200
Validation auc = 0.685915
Epoch 201
Validation auc = 0.690610
Epoch 202
Validation auc = 0.675587
Epoch 203
Validation auc = 0.676526
Epoch 204
Loss = 1.0649e-02, PNorm = 36.7431, GNorm = 0.0662, lr_0 = 7.1362e-04
Validation auc = 0.684038
Epoch 205
Validation auc = 0.700939
Epoch 206
Validation auc = 0.699061
Epoch 207
Validation auc = 0.687324
Epoch 208
Validation auc = 0.678404
Epoch 209
Loss = 8.4085e-02, PNorm = 36.7811, GNorm = 3.0213, lr_0 = 7.0752e-04
Validation auc = 0.682160
Epoch 210
Validation auc = 0.667136
Epoch 211
Validation auc = 0.657277
Epoch 212
Validation auc = 0.659624
Epoch 213
Validation auc = 0.653991
Epoch 214
Loss = 5.4911e-02, PNorm = 36.8184, GNorm = 1.6411, lr_0 = 7.0147e-04
Validation auc = 0.661972
Epoch 215
Validation auc = 0.677465
Epoch 216
Validation auc = 0.686854
Epoch 217
Validation auc = 0.688263
Epoch 218
Validation auc = 0.686385
Epoch 219
Loss = 1.6309e-02, PNorm = 36.8616, GNorm = 0.9491, lr_0 = 6.9548e-04
Validation auc = 0.683099
Epoch 220
Validation auc = 0.691080
Epoch 221
Validation auc = 0.679812
Epoch 222
Validation auc = 0.669014
Epoch 223
Validation auc = 0.676995
Epoch 224
Loss = 2.1156e-02, PNorm = 36.8990, GNorm = 1.1701, lr_0 = 6.8953e-04
Validation auc = 0.682160
Epoch 225
Validation auc = 0.692958
Epoch 226
Validation auc = 0.679343
Epoch 227
Validation auc = 0.683099
Epoch 228
Validation auc = 0.692019
Epoch 229
Loss = 3.9633e-02, PNorm = 36.9387, GNorm = 1.5712, lr_0 = 6.8364e-04
Validation auc = 0.686385
Epoch 230
Validation auc = 0.677934
Epoch 231
Validation auc = 0.672770
Epoch 232
Validation auc = 0.683568
Epoch 233
Validation auc = 0.686854
Epoch 234
Loss = 1.5093e-02, PNorm = 36.9816, GNorm = 0.3378, lr_0 = 6.7779e-04
Validation auc = 0.682160
Epoch 235
Validation auc = 0.677934
Epoch 236
Validation auc = 0.687793
Epoch 237
Validation auc = 0.686854
Epoch 238
Validation auc = 0.693427
Epoch 239
Loss = 2.2232e-02, PNorm = 37.0201, GNorm = 1.1330, lr_0 = 6.7200e-04
Validation auc = 0.686854
Epoch 240
Validation auc = 0.679343
Epoch 241
Validation auc = 0.670423
Epoch 242
Validation auc = 0.665728
Epoch 243
Validation auc = 0.669484
Epoch 244
Loss = 3.1850e-02, PNorm = 37.0562, GNorm = 1.8497, lr_0 = 6.6625e-04
Validation auc = 0.688732
Epoch 245
Validation auc = 0.697183
Epoch 246
Validation auc = 0.688732
Epoch 247
Validation auc = 0.687324
Epoch 248
Validation auc = 0.700939
Epoch 249
Loss = 5.5683e-02, PNorm = 37.0897, GNorm = 2.3111, lr_0 = 6.6056e-04
Validation auc = 0.683099
Epoch 250
Validation auc = 0.665728
Epoch 251
Validation auc = 0.673239
Epoch 252
Validation auc = 0.675587
Epoch 253
Validation auc = 0.665258
Epoch 254
Loss = 3.2694e-02, PNorm = 37.1264, GNorm = 1.4590, lr_0 = 6.5491e-04
Validation auc = 0.667606
Epoch 255
Validation auc = 0.673709
Epoch 256
Validation auc = 0.677465
Epoch 257
Validation auc = 0.680282
Epoch 258
Validation auc = 0.684507
Epoch 259
Loss = 1.6629e-02, PNorm = 37.1617, GNorm = 0.5041, lr_0 = 6.4931e-04
Validation auc = 0.682160
Epoch 260
Validation auc = 0.680282
Epoch 261
Validation auc = 0.673239
Epoch 262
Validation auc = 0.676056
Epoch 263
Validation auc = 0.689202
Epoch 264
Loss = 2.2011e-02, PNorm = 37.1929, GNorm = 2.1422, lr_0 = 6.4376e-04
Validation auc = 0.692019
Epoch 265
Validation auc = 0.681690
Epoch 266
Validation auc = 0.679343
Epoch 267
Validation auc = 0.684038
Epoch 268
Validation auc = 0.675117
Epoch 269
Loss = 1.5515e-02, PNorm = 37.2191, GNorm = 0.7171, lr_0 = 6.3826e-04
Validation auc = 0.662441
Epoch 270
Validation auc = 0.666197
Epoch 271
Validation auc = 0.677934
Epoch 272
Validation auc = 0.687793
Epoch 273
Validation auc = 0.686854
Epoch 274
Loss = 1.0354e-02, PNorm = 37.2535, GNorm = 0.1604, lr_0 = 6.3280e-04
Validation auc = 0.670892
Epoch 275
Validation auc = 0.667136
Epoch 276
Validation auc = 0.669014
Epoch 277
Validation auc = 0.666667
Epoch 278
Validation auc = 0.663380
Epoch 279
Loss = 8.6082e-03, PNorm = 37.2825, GNorm = 0.4271, lr_0 = 6.2739e-04
Validation auc = 0.661972
Epoch 280
Validation auc = 0.668075
Epoch 281
Validation auc = 0.677465
Epoch 282
Validation auc = 0.677465
Epoch 283
Validation auc = 0.673709
Epoch 284
Loss = 9.5378e-03, PNorm = 37.3103, GNorm = 0.1904, lr_0 = 6.2203e-04
Validation auc = 0.674648
Epoch 285
Validation auc = 0.676995
Epoch 286
Validation auc = 0.685446
Epoch 287
Validation auc = 0.683568
Epoch 288
Validation auc = 0.686854
Epoch 289
Loss = 1.0671e-02, PNorm = 37.3337, GNorm = 0.3591, lr_0 = 6.1671e-04
Validation auc = 0.687324
Epoch 290
Validation auc = 0.689202
Epoch 291
Validation auc = 0.686854
Epoch 292
Validation auc = 0.681690
Epoch 293
Validation auc = 0.685915
Epoch 294
Loss = 8.3785e-03, PNorm = 37.3567, GNorm = 0.3779, lr_0 = 6.1144e-04
Validation auc = 0.686854
Epoch 295
Validation auc = 0.675587
Epoch 296
Validation auc = 0.670423
Epoch 297
Validation auc = 0.674178
Epoch 298
Validation auc = 0.675117
Epoch 299
Loss = 8.4823e-03, PNorm = 37.3782, GNorm = 0.6124, lr_0 = 6.0621e-04
Validation auc = 0.669953
Model 0 best validation auc = 0.717371 on epoch 39
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.915111
Ensemble test auc = 0.915111
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=400, bias=False)
        (W_h): Linear(in_features=400, out_features=400, bias=False)
        (W_o): Linear(in_features=533, out_features=400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=628, out_features=1, bias=True)
  )
)
Number of parameters = 433,029
Moving model to cuda
Epoch 0
Validation auc = 0.396714
Epoch 1
Validation auc = 0.413615
Epoch 2
Validation auc = 0.432394
Epoch 3
Validation auc = 0.459155
Epoch 4
Loss = 7.2662e-01, PNorm = 32.8290, GNorm = 1.1976, lr_0 = 6.5000e-04
Validation auc = 0.470423
Epoch 5
Validation auc = 0.511268
Epoch 6
Validation auc = 0.561502
Epoch 7
Validation auc = 0.592958
Epoch 8
Validation auc = 0.607981
Epoch 9
Loss = 5.2963e-01, PNorm = 32.9209, GNorm = 2.6058, lr_0 = 9.9743e-04
Validation auc = 0.638498
Epoch 10
Validation auc = 0.634742
Epoch 11
Validation auc = 0.659624
Epoch 12
Validation auc = 0.666197
Epoch 13
Validation auc = 0.610798
Epoch 14
Loss = 5.7640e-01, PNorm = 32.9939, GNorm = 1.7523, lr_0 = 9.8890e-04
Validation auc = 0.592958
Epoch 15
Validation auc = 0.589202
Epoch 16
Validation auc = 0.589202
Epoch 17
Validation auc = 0.597653
Epoch 18
Validation auc = 0.608451
Epoch 19
Loss = 5.1373e-01, PNorm = 33.0695, GNorm = 1.0923, lr_0 = 9.8045e-04
Validation auc = 0.629108
Epoch 20
Validation auc = 0.639906
Epoch 21
Validation auc = 0.649765
Epoch 22
Validation auc = 0.658216
Epoch 23
Validation auc = 0.675587
Epoch 24
Loss = 4.7990e-01, PNorm = 33.1690, GNorm = 2.1450, lr_0 = 9.7207e-04
Validation auc = 0.647887
Epoch 25
Validation auc = 0.639906
Epoch 26
Validation auc = 0.646479
Epoch 27
Validation auc = 0.640376
Epoch 28
Validation auc = 0.655399
Epoch 29
Loss = 5.0760e-01, PNorm = 33.2699, GNorm = 4.7476, lr_0 = 9.6376e-04
Validation auc = 0.667136
Epoch 30
Validation auc = 0.648357
Epoch 31
Validation auc = 0.659624
Epoch 32
Validation auc = 0.667606
Epoch 33
Validation auc = 0.641315
Epoch 34
Loss = 4.3298e-01, PNorm = 33.3793, GNorm = 1.0566, lr_0 = 9.5552e-04
Validation auc = 0.670892
Epoch 35
Validation auc = 0.667136
Epoch 36
Validation auc = 0.651643
Epoch 37
Validation auc = 0.672770
Epoch 38
Validation auc = 0.694366
Epoch 39
Loss = 3.6739e-01, PNorm = 33.5057, GNorm = 1.3290, lr_0 = 9.4735e-04
Validation auc = 0.677465
Epoch 40
Validation auc = 0.694366
Epoch 41
Validation auc = 0.695305
Epoch 42
Validation auc = 0.680282
Epoch 43
Validation auc = 0.697183
Epoch 44
Loss = 3.6320e-01, PNorm = 33.6385, GNorm = 1.7008, lr_0 = 9.3925e-04
Validation auc = 0.669484
Epoch 45
Validation auc = 0.700469
Epoch 46
Validation auc = 0.690610
Epoch 47
Validation auc = 0.679343
Epoch 48
Validation auc = 0.693897
Epoch 49
Loss = 3.5429e-01, PNorm = 33.7843, GNorm = 1.2659, lr_0 = 9.3122e-04
Validation auc = 0.686385
Epoch 50
Validation auc = 0.673239
Epoch 51
Validation auc = 0.724883
Epoch 52
Validation auc = 0.678404
Epoch 53
Validation auc = 0.722535
Epoch 54
Loss = 3.9496e-01, PNorm = 33.9196, GNorm = 1.8608, lr_0 = 9.2326e-04
Validation auc = 0.668075
Epoch 55
Validation auc = 0.654930
Epoch 56
Validation auc = 0.676995
Epoch 57
Validation auc = 0.676526
Epoch 58
Validation auc = 0.651643
Epoch 59
Loss = 3.9418e-01, PNorm = 34.0472, GNorm = 2.5231, lr_0 = 9.1537e-04
Validation auc = 0.651643
Epoch 60
Validation auc = 0.673239
Epoch 61
Validation auc = 0.679343
Epoch 62
Validation auc = 0.685915
Epoch 63
Validation auc = 0.700000
Epoch 64
Loss = 2.3675e-01, PNorm = 34.1924, GNorm = 0.7548, lr_0 = 9.0754e-04
Validation auc = 0.702817
Epoch 65
Validation auc = 0.689671
Epoch 66
Validation auc = 0.681221
Epoch 67
Validation auc = 0.675587
Epoch 68
Validation auc = 0.664319
Epoch 69
Loss = 2.0511e-01, PNorm = 34.3306, GNorm = 0.6619, lr_0 = 8.9978e-04
Validation auc = 0.707042
Epoch 70
Validation auc = 0.690141
Epoch 71
Validation auc = 0.668545
Epoch 72
Validation auc = 0.683099
Epoch 73
Validation auc = 0.675117
Epoch 74
Loss = 2.4152e-01, PNorm = 34.4570, GNorm = 1.3034, lr_0 = 8.9209e-04
Validation auc = 0.672770
Epoch 75
Validation auc = 0.693897
Epoch 76
Validation auc = 0.706103
Epoch 77
Validation auc = 0.681690
Epoch 78
Validation auc = 0.675117
Epoch 79
Loss = 2.1231e-01, PNorm = 34.5874, GNorm = 1.0289, lr_0 = 8.8447e-04
Validation auc = 0.707981
Epoch 80
Validation auc = 0.706103
Epoch 81
Validation auc = 0.675587
Epoch 82
Validation auc = 0.679343
Epoch 83
Validation auc = 0.701408
Epoch 84
Loss = 1.9724e-01, PNorm = 34.7266, GNorm = 0.7694, lr_0 = 8.7691e-04
Validation auc = 0.684038
Epoch 85
Validation auc = 0.698122
Epoch 86
Validation auc = 0.711737
Epoch 87
Validation auc = 0.699061
Epoch 88
Validation auc = 0.696714
Epoch 89
Loss = 1.1522e-01, PNorm = 34.8657, GNorm = 0.7782, lr_0 = 8.6941e-04
Validation auc = 0.707042
Epoch 90
Validation auc = 0.698592
Epoch 91
Validation auc = 0.694366
Epoch 92
Validation auc = 0.693427
Epoch 93
Validation auc = 0.697183
Epoch 94
Loss = 1.7518e-01, PNorm = 34.9946, GNorm = 1.7117, lr_0 = 8.6198e-04
Validation auc = 0.702817
Epoch 95
Validation auc = 0.688732
Epoch 96
Validation auc = 0.704695
Epoch 97
Validation auc = 0.699531
Epoch 98
Validation auc = 0.683568
Epoch 99
Loss = 1.2474e-01, PNorm = 35.1132, GNorm = 2.9935, lr_0 = 8.5461e-04
Validation auc = 0.692488
Epoch 100
Validation auc = 0.673239
Epoch 101
Validation auc = 0.692958
Epoch 102
Validation auc = 0.702347
Epoch 103
Validation auc = 0.692488
Epoch 104
Loss = 1.0005e-01, PNorm = 35.2311, GNorm = 1.0066, lr_0 = 8.4730e-04
Validation auc = 0.688263
Epoch 105
Validation auc = 0.681690
Epoch 106
Validation auc = 0.671831
Epoch 107
Validation auc = 0.670423
Epoch 108
Validation auc = 0.715023
Epoch 109
Loss = 2.4621e-01, PNorm = 35.3331, GNorm = 1.7592, lr_0 = 8.4006e-04
Validation auc = 0.648826
Epoch 110
Validation auc = 0.656338
Epoch 111
Validation auc = 0.677465
Epoch 112
Validation auc = 0.675117
Epoch 113
Validation auc = 0.661502
Epoch 114
Loss = 1.8243e-01, PNorm = 35.4387, GNorm = 2.7253, lr_0 = 8.3288e-04
Validation auc = 0.681690
Epoch 115
Validation auc = 0.707512
Epoch 116
Validation auc = 0.690141
Epoch 117
Validation auc = 0.668075
Epoch 118
Validation auc = 0.662441
Epoch 119
Loss = 2.3081e-01, PNorm = 35.5450, GNorm = 1.0502, lr_0 = 8.2576e-04
Validation auc = 0.673709
Epoch 120
Validation auc = 0.666667
Epoch 121
Validation auc = 0.660094
Epoch 122
Validation auc = 0.679812
Epoch 123
Validation auc = 0.704695
Epoch 124
Loss = 1.1245e-01, PNorm = 35.6511, GNorm = 1.6538, lr_0 = 8.1870e-04
Validation auc = 0.706103
Epoch 125
Validation auc = 0.694836
Epoch 126
Validation auc = 0.700000
Epoch 127
Validation auc = 0.714554
Epoch 128
Validation auc = 0.697183
Epoch 129
Loss = 6.9323e-02, PNorm = 35.7370, GNorm = 0.3759, lr_0 = 8.1170e-04
Validation auc = 0.679343
Epoch 130
Validation auc = 0.692958
Epoch 131
Validation auc = 0.706573
Epoch 132
Validation auc = 0.692958
Epoch 133
Validation auc = 0.677465
Epoch 134
Loss = 1.1878e-01, PNorm = 35.8194, GNorm = 1.0521, lr_0 = 8.0476e-04
Validation auc = 0.680751
Epoch 135
Validation auc = 0.695305
Epoch 136
Validation auc = 0.698122
Epoch 137
Validation auc = 0.688732
Epoch 138
Validation auc = 0.695775
Epoch 139
Loss = 7.3437e-02, PNorm = 35.9066, GNorm = 0.3916, lr_0 = 7.9788e-04
Validation auc = 0.710329
Epoch 140
Validation auc = 0.715023
Epoch 141
Validation auc = 0.701408
Epoch 142
Validation auc = 0.681690
Epoch 143
Validation auc = 0.684977
Epoch 144
Loss = 9.2404e-02, PNorm = 35.9829, GNorm = 1.5883, lr_0 = 7.9106e-04
Validation auc = 0.684507
Epoch 145
Validation auc = 0.673709
Epoch 146
Validation auc = 0.669953
Epoch 147
Validation auc = 0.676995
Epoch 148
Validation auc = 0.679343
Epoch 149
Loss = 7.0173e-02, PNorm = 36.0720, GNorm = 0.8375, lr_0 = 7.8430e-04
Validation auc = 0.671831
Epoch 150
Validation auc = 0.669953
Epoch 151
Validation auc = 0.678873
Epoch 152
Validation auc = 0.683099
Epoch 153
Validation auc = 0.684507
Epoch 154
Loss = 3.8213e-02, PNorm = 36.1594, GNorm = 0.7851, lr_0 = 7.7759e-04
Validation auc = 0.683099
Epoch 155
Validation auc = 0.700469
Epoch 156
Validation auc = 0.696714
Epoch 157
Validation auc = 0.695775
Epoch 158
Validation auc = 0.700939
Epoch 159
Loss = 5.3124e-02, PNorm = 36.2443, GNorm = 0.7175, lr_0 = 7.7095e-04
Validation auc = 0.689202
Epoch 160
Validation auc = 0.679812
Epoch 161
Validation auc = 0.682629
Epoch 162
Validation auc = 0.693897
Epoch 163
Validation auc = 0.678873
Epoch 164
Loss = 8.8586e-02, PNorm = 36.3185, GNorm = 1.8722, lr_0 = 7.6436e-04
Validation auc = 0.677465
Epoch 165
Validation auc = 0.701878
Epoch 166
Validation auc = 0.688263
Epoch 167
Validation auc = 0.668075
Epoch 168
Validation auc = 0.670892
Epoch 169
Loss = 7.5254e-02, PNorm = 36.3979, GNorm = 0.9176, lr_0 = 7.5782e-04
Validation auc = 0.681690
Epoch 170
Validation auc = 0.674178
Epoch 171
Validation auc = 0.669484
Epoch 172
Validation auc = 0.680282
Epoch 173
Validation auc = 0.691549
Epoch 174
Loss = 4.2540e-02, PNorm = 36.4712, GNorm = 1.0212, lr_0 = 7.5134e-04
Validation auc = 0.690141
Epoch 175
Validation auc = 0.685915
Epoch 176
Validation auc = 0.684038
Epoch 177
Validation auc = 0.693427
Epoch 178
Validation auc = 0.693897
Epoch 179
Loss = 4.0722e-02, PNorm = 36.5375, GNorm = 0.2291, lr_0 = 7.4492e-04
Validation auc = 0.685446
Epoch 180
Validation auc = 0.682629
Epoch 181
Validation auc = 0.687324
Epoch 182
Validation auc = 0.684507
Epoch 183
Validation auc = 0.675117
Epoch 184
Loss = 7.1085e-02, PNorm = 36.6019, GNorm = 1.3376, lr_0 = 7.3855e-04
Validation auc = 0.687324
Epoch 185
Validation auc = 0.689671
Epoch 186
Validation auc = 0.676056
Epoch 187
Validation auc = 0.681690
Epoch 188
Validation auc = 0.693897
Epoch 189
Loss = 3.9399e-02, PNorm = 36.6659, GNorm = 0.7890, lr_0 = 7.3224e-04
Validation auc = 0.692019
Epoch 190
Validation auc = 0.684977
Epoch 191
Validation auc = 0.690141
Epoch 192
Validation auc = 0.680282
Epoch 193
Validation auc = 0.667136
Epoch 194
Loss = 8.6966e-02, PNorm = 36.7246, GNorm = 0.9237, lr_0 = 7.2598e-04
Validation auc = 0.691080
Epoch 195
Validation auc = 0.691080
Epoch 196
Validation auc = 0.671831
Epoch 197
Validation auc = 0.680282
Epoch 198
Validation auc = 0.684507
Epoch 199
Loss = 2.8712e-02, PNorm = 36.7945, GNorm = 0.2739, lr_0 = 7.1977e-04
Validation auc = 0.678404
Epoch 200
Validation auc = 0.674648
Epoch 201
Validation auc = 0.680282
Epoch 202
Validation auc = 0.698592
Epoch 203
Validation auc = 0.687793
Epoch 204
Loss = 2.0155e-02, PNorm = 36.8579, GNorm = 1.0508, lr_0 = 7.1362e-04
Validation auc = 0.668545
Epoch 205
Validation auc = 0.699061
Epoch 206
Validation auc = 0.686385
Epoch 207
Validation auc = 0.661033
Epoch 208
Validation auc = 0.674648
Epoch 209
Loss = 7.5725e-02, PNorm = 36.9321, GNorm = 4.2723, lr_0 = 7.0752e-04
Validation auc = 0.676056
Epoch 210
Validation auc = 0.655869
Epoch 211
Validation auc = 0.646479
Epoch 212
Validation auc = 0.664319
Epoch 213
Validation auc = 0.687324
Epoch 214
Loss = 9.2641e-02, PNorm = 37.0091, GNorm = 1.9156, lr_0 = 7.0147e-04
Validation auc = 0.694366
Epoch 215
Validation auc = 0.685915
Epoch 216
Validation auc = 0.685915
Epoch 217
Validation auc = 0.697653
Epoch 218
Validation auc = 0.707512
Epoch 219
Loss = 4.7924e-02, PNorm = 37.0835, GNorm = 1.1221, lr_0 = 6.9548e-04
Validation auc = 0.692958
Epoch 220
Validation auc = 0.676056
Epoch 221
Validation auc = 0.666667
Epoch 222
Validation auc = 0.661972
Epoch 223
Validation auc = 0.664789
Epoch 224
Loss = 2.8510e-02, PNorm = 37.1420, GNorm = 0.5524, lr_0 = 6.8953e-04
Validation auc = 0.665258
Epoch 225
Validation auc = 0.670892
Epoch 226
Validation auc = 0.666667
Epoch 227
Validation auc = 0.663850
Epoch 228
Validation auc = 0.677465
Epoch 229
Loss = 5.1703e-02, PNorm = 37.2000, GNorm = 1.8703, lr_0 = 6.8364e-04
Validation auc = 0.692958
Epoch 230
Validation auc = 0.668075
Epoch 231
Validation auc = 0.647887
Epoch 232
Validation auc = 0.649296
Epoch 233
Validation auc = 0.661033
Epoch 234
Loss = 3.1598e-02, PNorm = 37.2531, GNorm = 1.6009, lr_0 = 6.7779e-04
Validation auc = 0.664319
Epoch 235
Validation auc = 0.662441
Epoch 236
Validation auc = 0.659155
Epoch 237
Validation auc = 0.668545
Epoch 238
Validation auc = 0.676526
Epoch 239
Loss = 2.7112e-02, PNorm = 37.3055, GNorm = 0.6787, lr_0 = 6.7200e-04
Validation auc = 0.693897
Epoch 240
Validation auc = 0.698122
Epoch 241
Validation auc = 0.691549
Epoch 242
Validation auc = 0.693897
Epoch 243
Validation auc = 0.703286
Epoch 244
Loss = 3.4391e-02, PNorm = 37.3508, GNorm = 2.0579, lr_0 = 6.6625e-04
Validation auc = 0.703286
Epoch 245
Validation auc = 0.684038
Epoch 246
Validation auc = 0.671831
Epoch 247
Validation auc = 0.666197
Epoch 248
Validation auc = 0.657746
Epoch 249
Loss = 4.3628e-02, PNorm = 37.3927, GNorm = 1.6688, lr_0 = 6.6056e-04
Validation auc = 0.664789
Epoch 250
Validation auc = 0.679343
Epoch 251
Validation auc = 0.678873
Epoch 252
Validation auc = 0.677465
Epoch 253
Validation auc = 0.680751
Epoch 254
Loss = 2.2271e-02, PNorm = 37.4399, GNorm = 1.0392, lr_0 = 6.5491e-04
Validation auc = 0.686854
Epoch 255
Validation auc = 0.697183
Epoch 256
Validation auc = 0.695775
Epoch 257
Validation auc = 0.680751
Epoch 258
Validation auc = 0.664789
Epoch 259
Loss = 1.8575e-02, PNorm = 37.4805, GNorm = 1.0663, lr_0 = 6.4931e-04
Validation auc = 0.660094
Epoch 260
Validation auc = 0.666667
Epoch 261
Validation auc = 0.668075
Epoch 262
Validation auc = 0.658685
Epoch 263
Validation auc = 0.656808
Epoch 264
Loss = 4.4659e-02, PNorm = 37.5222, GNorm = 0.1446, lr_0 = 6.4376e-04
Validation auc = 0.668075
Epoch 265
Validation auc = 0.673709
Epoch 266
Validation auc = 0.676995
Epoch 267
Validation auc = 0.675117
Epoch 268
Validation auc = 0.669014
Epoch 269
Loss = 2.5692e-02, PNorm = 37.5570, GNorm = 1.1006, lr_0 = 6.3826e-04
Validation auc = 0.669953
Epoch 270
Validation auc = 0.676526
Epoch 271
Validation auc = 0.673709
Epoch 272
Validation auc = 0.663850
Epoch 273
Validation auc = 0.661033
Epoch 274
Loss = 5.3998e-03, PNorm = 37.5972, GNorm = 0.1280, lr_0 = 6.3280e-04
Validation auc = 0.665728
Epoch 275
Validation auc = 0.679343
Epoch 276
Validation auc = 0.682629
Epoch 277
Validation auc = 0.674648
Epoch 278
Validation auc = 0.669014
Epoch 279
Loss = 1.6950e-02, PNorm = 37.6399, GNorm = 0.6950, lr_0 = 6.2739e-04
Validation auc = 0.667136
Epoch 280
Validation auc = 0.666667
Epoch 281
Validation auc = 0.663850
Epoch 282
Validation auc = 0.659624
Epoch 283
Validation auc = 0.660563
Epoch 284
Loss = 1.2343e-02, PNorm = 37.6752, GNorm = 0.2106, lr_0 = 6.2203e-04
Validation auc = 0.670423
Epoch 285
Validation auc = 0.676995
Epoch 286
Validation auc = 0.676526
Epoch 287
Validation auc = 0.682160
Epoch 288
Validation auc = 0.684038
Epoch 289
Loss = 5.6120e-03, PNorm = 37.7100, GNorm = 0.2184, lr_0 = 6.1671e-04
Validation auc = 0.691080
Epoch 290
Validation auc = 0.692958
Epoch 291
Validation auc = 0.694836
Epoch 292
Validation auc = 0.694836
Epoch 293
Validation auc = 0.692958
Epoch 294
Loss = 7.8700e-03, PNorm = 37.7376, GNorm = 0.4379, lr_0 = 6.1144e-04
Validation auc = 0.691080
Epoch 295
Validation auc = 0.688263
Epoch 296
Validation auc = 0.689671
Epoch 297
Validation auc = 0.685915
Epoch 298
Validation auc = 0.678404
Epoch 299
Loss = 2.1788e-02, PNorm = 37.7607, GNorm = 1.1844, lr_0 = 6.0621e-04
Validation auc = 0.679812
Model 0 best validation auc = 0.724883 on epoch 51
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.888621
Ensemble test auc = 0.888621
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=400, bias=False)
        (W_h): Linear(in_features=400, out_features=400, bias=False)
        (W_o): Linear(in_features=533, out_features=400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=628, out_features=1, bias=True)
  )
)
Number of parameters = 433,029
Moving model to cuda
Epoch 0
Validation auc = 0.394366
Epoch 1
Validation auc = 0.410329
Epoch 2
Validation auc = 0.430047
Epoch 3
Validation auc = 0.451643
Epoch 4
Loss = 7.2197e-01, PNorm = 32.8288, GNorm = 1.5039, lr_0 = 6.5000e-04
Validation auc = 0.464319
Epoch 5
Validation auc = 0.506103
Epoch 6
Validation auc = 0.552582
Epoch 7
Validation auc = 0.573709
Epoch 8
Validation auc = 0.600939
Epoch 9
Loss = 6.1243e-01, PNorm = 32.9037, GNorm = 1.5287, lr_0 = 9.9743e-04
Validation auc = 0.619249
Epoch 10
Validation auc = 0.614085
Epoch 11
Validation auc = 0.651174
Epoch 12
Validation auc = 0.692488
Epoch 13
Validation auc = 0.696714
Epoch 14
Loss = 5.2593e-01, PNorm = 33.0010, GNorm = 2.9139, lr_0 = 9.8890e-04
Validation auc = 0.653991
Epoch 15
Validation auc = 0.634272
Epoch 16
Validation auc = 0.641315
Epoch 17
Validation auc = 0.626761
Epoch 18
Validation auc = 0.642254
Epoch 19
Loss = 5.1122e-01, PNorm = 33.0949, GNorm = 1.5080, lr_0 = 9.8045e-04
Validation auc = 0.684507
Epoch 20
Validation auc = 0.648357
Epoch 21
Validation auc = 0.646479
Epoch 22
Validation auc = 0.676056
Epoch 23
Validation auc = 0.640376
Epoch 24
Loss = 5.0823e-01, PNorm = 33.1961, GNorm = 2.5661, lr_0 = 9.7207e-04
Validation auc = 0.649296
Epoch 25
Validation auc = 0.677465
Epoch 26
Validation auc = 0.643662
Epoch 27
Validation auc = 0.656808
Epoch 28
Validation auc = 0.685915
Epoch 29
Loss = 4.3166e-01, PNorm = 33.3095, GNorm = 0.8534, lr_0 = 9.6376e-04
Validation auc = 0.676526
Epoch 30
Validation auc = 0.675587
Epoch 31
Validation auc = 0.705634
Epoch 32
Validation auc = 0.685446
Epoch 33
Validation auc = 0.703286
Epoch 34
Loss = 3.5664e-01, PNorm = 33.4459, GNorm = 1.7777, lr_0 = 9.5552e-04
Validation auc = 0.695305
Epoch 35
Validation auc = 0.705634
Epoch 36
Validation auc = 0.706103
Epoch 37
Validation auc = 0.666197
Epoch 38
Validation auc = 0.699531
Epoch 39
Loss = 4.2173e-01, PNorm = 33.5705, GNorm = 3.2410, lr_0 = 9.4735e-04
Validation auc = 0.700469
Epoch 40
Validation auc = 0.674648
Epoch 41
Validation auc = 0.717840
Epoch 42
Validation auc = 0.715493
Epoch 43
Validation auc = 0.676995
Epoch 44
Loss = 4.2003e-01, PNorm = 33.7078, GNorm = 2.2495, lr_0 = 9.3925e-04
Validation auc = 0.675587
Epoch 45
Validation auc = 0.682629
Epoch 46
Validation auc = 0.679343
Epoch 47
Validation auc = 0.671362
Epoch 48
Validation auc = 0.692488
Epoch 49
Loss = 2.2663e-01, PNorm = 33.8563, GNorm = 0.8505, lr_0 = 9.3122e-04
Validation auc = 0.697653
Epoch 50
Validation auc = 0.707512
Epoch 51
Validation auc = 0.707981
Epoch 52
Validation auc = 0.707042
Epoch 53
Validation auc = 0.710329
Epoch 54
Loss = 2.9741e-01, PNorm = 34.0159, GNorm = 3.3023, lr_0 = 9.2326e-04
Validation auc = 0.707512
Epoch 55
Validation auc = 0.703286
Epoch 56
Validation auc = 0.675587
Epoch 57
Validation auc = 0.680751
Epoch 58
Validation auc = 0.694366
Epoch 59
Loss = 1.9489e-01, PNorm = 34.1708, GNorm = 1.4379, lr_0 = 9.1537e-04
Validation auc = 0.688732
Epoch 60
Validation auc = 0.700469
Epoch 61
Validation auc = 0.719718
Epoch 62
Validation auc = 0.709390
Epoch 63
Validation auc = 0.718310
Epoch 64
Loss = 2.1760e-01, PNorm = 34.3334, GNorm = 1.4682, lr_0 = 9.0754e-04
Validation auc = 0.707512
Epoch 65
Validation auc = 0.705634
Epoch 66
Validation auc = 0.664789
Epoch 67
Validation auc = 0.705164
Epoch 68
Validation auc = 0.691549
Epoch 69
Loss = 2.1105e-01, PNorm = 34.4923, GNorm = 1.2476, lr_0 = 8.9978e-04
Validation auc = 0.707042
Epoch 70
Validation auc = 0.700939
Epoch 71
Validation auc = 0.708451
Epoch 72
Validation auc = 0.704695
Epoch 73
Validation auc = 0.684038
Epoch 74
Loss = 1.7721e-01, PNorm = 34.6451, GNorm = 1.4157, lr_0 = 8.9209e-04
Validation auc = 0.688263
Epoch 75
Validation auc = 0.677465
Epoch 76
Validation auc = 0.703286
Epoch 77
Validation auc = 0.688732
Epoch 78
Validation auc = 0.708920
Epoch 79
Loss = 1.0100e-01, PNorm = 34.7971, GNorm = 1.1640, lr_0 = 8.8447e-04
Validation auc = 0.685446
Epoch 80
Validation auc = 0.701878
Epoch 81
Validation auc = 0.677465
Epoch 82
Validation auc = 0.702347
Epoch 83
Validation auc = 0.673239
Epoch 84
Loss = 1.3755e-01, PNorm = 34.9396, GNorm = 3.1396, lr_0 = 8.7691e-04
Validation auc = 0.673239
Epoch 85
Validation auc = 0.671362
Epoch 86
Validation auc = 0.677465
Epoch 87
Validation auc = 0.661033
Epoch 88
Validation auc = 0.699531
Epoch 89
Loss = 1.5978e-01, PNorm = 35.0615, GNorm = 1.4228, lr_0 = 8.6941e-04
Validation auc = 0.688732
Epoch 90
Validation auc = 0.689671
Epoch 91
Validation auc = 0.691080
Epoch 92
Validation auc = 0.664789
Epoch 93
Validation auc = 0.700000
Epoch 94
Loss = 1.8599e-01, PNorm = 35.1726, GNorm = 1.7255, lr_0 = 8.6198e-04
Validation auc = 0.684977
Epoch 95
Validation auc = 0.693897
Epoch 96
Validation auc = 0.694366
Epoch 97
Validation auc = 0.676526
Epoch 98
Validation auc = 0.674648
Epoch 99
Loss = 9.9706e-02, PNorm = 35.2718, GNorm = 4.4197, lr_0 = 8.5461e-04
Validation auc = 0.657746
Epoch 100
Validation auc = 0.667136
Epoch 101
Validation auc = 0.664319
Epoch 102
Validation auc = 0.690610
Epoch 103
Validation auc = 0.708920
Epoch 104
Loss = 1.2777e-01, PNorm = 35.3650, GNorm = 2.7138, lr_0 = 8.4730e-04
Validation auc = 0.705634
Epoch 105
Validation auc = 0.710329
Epoch 106
Validation auc = 0.688732
Epoch 107
Validation auc = 0.692958
Epoch 108
Validation auc = 0.677465
Epoch 109
Loss = 5.5738e-02, PNorm = 35.4534, GNorm = 0.6758, lr_0 = 8.4006e-04
Validation auc = 0.666197
Epoch 110
Validation auc = 0.696244
Epoch 111
Validation auc = 0.686854
Epoch 112
Validation auc = 0.712207
Epoch 113
Validation auc = 0.717840
Epoch 114
Loss = 9.2100e-02, PNorm = 35.5377, GNorm = 2.8085, lr_0 = 8.3288e-04
Validation auc = 0.697183
Epoch 115
Validation auc = 0.694366
Epoch 116
Validation auc = 0.684507
Epoch 117
Validation auc = 0.690610
Epoch 118
Validation auc = 0.716901
Epoch 119
Loss = 7.9807e-02, PNorm = 35.6205, GNorm = 2.3026, lr_0 = 8.2576e-04
Validation auc = 0.696244
Epoch 120
Validation auc = 0.701878
Epoch 121
Validation auc = 0.714085
Epoch 122
Validation auc = 0.694366
Epoch 123
Validation auc = 0.697653
Epoch 124
Loss = 6.1261e-02, PNorm = 35.7065, GNorm = 0.5182, lr_0 = 8.1870e-04
Validation auc = 0.692488
Epoch 125
Validation auc = 0.700469
Epoch 126
Validation auc = 0.703756
Epoch 127
Validation auc = 0.706573
Epoch 128
Validation auc = 0.700000
Epoch 129
Loss = 5.5651e-02, PNorm = 35.7923, GNorm = 0.8197, lr_0 = 8.1170e-04
Validation auc = 0.692488
Epoch 130
Validation auc = 0.695305
Epoch 131
Validation auc = 0.673709
Epoch 132
Validation auc = 0.678873
Epoch 133
Validation auc = 0.680282
Epoch 134
Loss = 5.7477e-02, PNorm = 35.8753, GNorm = 1.2759, lr_0 = 8.0476e-04
Validation auc = 0.677465
Epoch 135
Validation auc = 0.700939
Epoch 136
Validation auc = 0.706573
Epoch 137
Validation auc = 0.696714
Epoch 138
Validation auc = 0.720188
Epoch 139
Loss = 1.0455e-01, PNorm = 35.9540, GNorm = 0.6643, lr_0 = 7.9788e-04
Validation auc = 0.694366
Epoch 140
Validation auc = 0.689671
Epoch 141
Validation auc = 0.713146
Epoch 142
Validation auc = 0.699531
Epoch 143
Validation auc = 0.685915
Epoch 144
Loss = 1.1274e-01, PNorm = 36.0412, GNorm = 0.7691, lr_0 = 7.9106e-04
Validation auc = 0.703286
Epoch 145
Validation auc = 0.700000
Epoch 146
Validation auc = 0.685446
Epoch 147
Validation auc = 0.683099
Epoch 148
Validation auc = 0.705164
Epoch 149
Loss = 1.0204e-01, PNorm = 36.1196, GNorm = 1.7823, lr_0 = 7.8430e-04
Validation auc = 0.699061
Epoch 150
Validation auc = 0.685915
Epoch 151
Validation auc = 0.684977
Epoch 152
Validation auc = 0.699531
Epoch 153
Validation auc = 0.643662
Epoch 154
Loss = 1.0157e-01, PNorm = 36.1996, GNorm = 3.4872, lr_0 = 7.7759e-04
Validation auc = 0.652582
Epoch 155
Validation auc = 0.688732
Epoch 156
Validation auc = 0.702817
Epoch 157
Validation auc = 0.693897
Epoch 158
Validation auc = 0.699531
Epoch 159
Loss = 2.8255e-02, PNorm = 36.2853, GNorm = 0.8919, lr_0 = 7.7095e-04
Validation auc = 0.710798
Epoch 160
Validation auc = 0.720657
Epoch 161
Validation auc = 0.722535
Epoch 162
Validation auc = 0.712207
Epoch 163
Validation auc = 0.710798
Epoch 164
Loss = 5.2420e-02, PNorm = 36.3581, GNorm = 0.3547, lr_0 = 7.6436e-04
Validation auc = 0.710329
Epoch 165
Validation auc = 0.707981
Epoch 166
Validation auc = 0.701878
Epoch 167
Validation auc = 0.703286
Epoch 168
Validation auc = 0.699531
Epoch 169
Loss = 3.4184e-02, PNorm = 36.4212, GNorm = 0.4639, lr_0 = 7.5782e-04
Validation auc = 0.697653
Epoch 170
Validation auc = 0.701878
Epoch 171
Validation auc = 0.700469
Epoch 172
Validation auc = 0.712676
Epoch 173
Validation auc = 0.694366
Epoch 174
Loss = 2.5660e-02, PNorm = 36.4735, GNorm = 0.5773, lr_0 = 7.5134e-04
Validation auc = 0.680282
Epoch 175
Validation auc = 0.693897
Epoch 176
Validation auc = 0.695775
Epoch 177
Validation auc = 0.681221
Epoch 178
Validation auc = 0.669014
Epoch 179
Loss = 3.1746e-02, PNorm = 36.5244, GNorm = 0.4931, lr_0 = 7.4492e-04
Validation auc = 0.683568
Epoch 180
Validation auc = 0.695305
Epoch 181
Validation auc = 0.683099
Epoch 182
Validation auc = 0.678873
Epoch 183
Validation auc = 0.692488
Epoch 184
Loss = 3.1469e-02, PNorm = 36.5758, GNorm = 1.1673, lr_0 = 7.3855e-04
Validation auc = 0.689671
Epoch 185
Validation auc = 0.677465
Epoch 186
Validation auc = 0.687324
Epoch 187
Validation auc = 0.693897
Epoch 188
Validation auc = 0.685915
Epoch 189
Loss = 1.5556e-02, PNorm = 36.6308, GNorm = 0.2380, lr_0 = 7.3224e-04
Validation auc = 0.679812
Epoch 190
Validation auc = 0.680751
Epoch 191
Validation auc = 0.705634
Epoch 192
Validation auc = 0.677465
Epoch 193
Validation auc = 0.667606
Epoch 194
Loss = 4.4259e-02, PNorm = 36.6854, GNorm = 1.8986, lr_0 = 7.2598e-04
Validation auc = 0.696244
Epoch 195
Validation auc = 0.698592
Epoch 196
Validation auc = 0.683099
Epoch 197
Validation auc = 0.670892
Epoch 198
Validation auc = 0.684038
Epoch 199
Loss = 3.3382e-02, PNorm = 36.7479, GNorm = 0.4005, lr_0 = 7.1977e-04
Validation auc = 0.706573
Epoch 200
Validation auc = 0.701878
Epoch 201
Validation auc = 0.688732
Epoch 202
Validation auc = 0.687324
Epoch 203
Validation auc = 0.689202
Epoch 204
Loss = 2.4368e-02, PNorm = 36.7964, GNorm = 0.7421, lr_0 = 7.1362e-04
Validation auc = 0.685915
Epoch 205
Validation auc = 0.675587
Epoch 206
Validation auc = 0.677465
Epoch 207
Validation auc = 0.691080
Epoch 208
Validation auc = 0.691080
Epoch 209
Loss = 1.3543e-02, PNorm = 36.8508, GNorm = 0.0869, lr_0 = 7.0752e-04
Validation auc = 0.681221
Epoch 210
Validation auc = 0.695775
Epoch 211
Validation auc = 0.705164
Epoch 212
Validation auc = 0.707512
Epoch 213
Validation auc = 0.699531
Epoch 214
Loss = 1.4321e-02, PNorm = 36.8957, GNorm = 0.5819, lr_0 = 7.0147e-04
Validation auc = 0.696244
Epoch 215
Validation auc = 0.694366
Epoch 216
Validation auc = 0.709859
Epoch 217
Validation auc = 0.706103
Epoch 218
Validation auc = 0.694366
Epoch 219
Loss = 1.5708e-02, PNorm = 36.9414, GNorm = 0.0850, lr_0 = 6.9548e-04
Validation auc = 0.690610
Epoch 220
Validation auc = 0.699061
Epoch 221
Validation auc = 0.699061
Epoch 222
Validation auc = 0.683568
Epoch 223
Validation auc = 0.677465
Epoch 224
Loss = 2.2270e-02, PNorm = 36.9883, GNorm = 0.4233, lr_0 = 6.8953e-04
Validation auc = 0.694836
Epoch 225
Validation auc = 0.692488
Epoch 226
Validation auc = 0.674178
Epoch 227
Validation auc = 0.670423
Epoch 228
Validation auc = 0.673709
Epoch 229
Loss = 1.9756e-02, PNorm = 37.0310, GNorm = 0.2766, lr_0 = 6.8364e-04
Validation auc = 0.685446
Epoch 230
Validation auc = 0.687324
Epoch 231
Validation auc = 0.681221
Epoch 232
Validation auc = 0.685915
Epoch 233
Validation auc = 0.699061
Epoch 234
Loss = 3.4032e-02, PNorm = 37.0748, GNorm = 2.7393, lr_0 = 6.7779e-04
Validation auc = 0.692958
Epoch 235
Validation auc = 0.676526
Epoch 236
Validation auc = 0.679812
Epoch 237
Validation auc = 0.685915
Epoch 238
Validation auc = 0.687324
Epoch 239
Loss = 1.3555e-02, PNorm = 37.1152, GNorm = 0.2883, lr_0 = 6.7200e-04
Validation auc = 0.683099
Epoch 240
Validation auc = 0.687793
Epoch 241
Validation auc = 0.699061
Epoch 242
Validation auc = 0.704225
Epoch 243
Validation auc = 0.699061
Epoch 244
Loss = 8.3409e-03, PNorm = 37.1527, GNorm = 0.4925, lr_0 = 6.6625e-04
Validation auc = 0.696244
Epoch 245
Validation auc = 0.699531
Epoch 246
Validation auc = 0.716432
Epoch 247
Validation auc = 0.698122
Epoch 248
Validation auc = 0.689671
Epoch 249
Loss = 1.4346e-02, PNorm = 37.1891, GNorm = 1.0919, lr_0 = 6.6056e-04
Validation auc = 0.690610
Epoch 250
Validation auc = 0.690610
Epoch 251
Validation auc = 0.684507
Epoch 252
Validation auc = 0.679343
Epoch 253
Validation auc = 0.681690
Epoch 254
Loss = 7.9013e-03, PNorm = 37.2245, GNorm = 0.4115, lr_0 = 6.5491e-04
Validation auc = 0.682629
Epoch 255
Validation auc = 0.675587
Epoch 256
Validation auc = 0.671831
Epoch 257
Validation auc = 0.681690
Epoch 258
Validation auc = 0.699061
Epoch 259
Loss = 2.4405e-02, PNorm = 37.2599, GNorm = 2.1497, lr_0 = 6.4931e-04
Validation auc = 0.689202
Epoch 260
Validation auc = 0.667136
Epoch 261
Validation auc = 0.666667
Epoch 262
Validation auc = 0.692488
Epoch 263
Validation auc = 0.695775
Epoch 264
Loss = 1.4379e-02, PNorm = 37.2936, GNorm = 0.6116, lr_0 = 6.4376e-04
Validation auc = 0.680751
Epoch 265
Validation auc = 0.686385
Epoch 266
Validation auc = 0.706573
Epoch 267
Validation auc = 0.700469
Epoch 268
Validation auc = 0.692488
Epoch 269
Loss = 9.4462e-02, PNorm = 37.3466, GNorm = 2.0594, lr_0 = 6.3826e-04
Validation auc = 0.702347
Epoch 270
Validation auc = 0.705634
Epoch 271
Validation auc = 0.678873
Epoch 272
Validation auc = 0.661033
Epoch 273
Validation auc = 0.657746
Epoch 274
Loss = 2.3358e-02, PNorm = 37.3996, GNorm = 1.1254, lr_0 = 6.3280e-04
Validation auc = 0.663380
Epoch 275
Validation auc = 0.676526
Epoch 276
Validation auc = 0.677465
Epoch 277
Validation auc = 0.672770
Epoch 278
Validation auc = 0.668545
Epoch 279
Loss = 8.7808e-03, PNorm = 37.4501, GNorm = 0.1877, lr_0 = 6.2739e-04
Validation auc = 0.670892
Epoch 280
Validation auc = 0.683568
Epoch 281
Validation auc = 0.693427
Epoch 282
Validation auc = 0.685446
Epoch 283
Validation auc = 0.676056
Epoch 284
Loss = 8.7833e-02, PNorm = 37.4926, GNorm = 2.0596, lr_0 = 6.2203e-04
Validation auc = 0.692019
Epoch 285
Validation auc = 0.702817
Epoch 286
Validation auc = 0.698122
Epoch 287
Validation auc = 0.691549
Epoch 288
Validation auc = 0.687324
Epoch 289
Loss = 1.8026e-02, PNorm = 37.5296, GNorm = 1.0334, lr_0 = 6.1671e-04
Validation auc = 0.691080
Epoch 290
Validation auc = 0.697653
Epoch 291
Validation auc = 0.691080
Epoch 292
Validation auc = 0.683568
Epoch 293
Validation auc = 0.675587
Epoch 294
Loss = 1.0089e-02, PNorm = 37.5607, GNorm = 0.4649, lr_0 = 6.1144e-04
Validation auc = 0.674178
Epoch 295
Validation auc = 0.687324
Epoch 296
Validation auc = 0.695305
Epoch 297
Validation auc = 0.692488
Epoch 298
Validation auc = 0.685915
Epoch 299
Loss = 2.3317e-03, PNorm = 37.5925, GNorm = 0.0515, lr_0 = 6.0621e-04
Validation auc = 0.684038
Model 0 best validation auc = 0.722535 on epoch 161
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.873570
Ensemble test auc = 0.873570
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=400, bias=False)
        (W_h): Linear(in_features=400, out_features=400, bias=False)
        (W_o): Linear(in_features=533, out_features=400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=628, out_features=1, bias=True)
  )
)
Number of parameters = 433,029
Moving model to cuda
Epoch 0
Validation auc = 0.396244
Epoch 1
Validation auc = 0.408920
Epoch 2
Validation auc = 0.426291
Epoch 3
Validation auc = 0.451643
Epoch 4
Loss = 7.8682e-01, PNorm = 32.8316, GNorm = 1.8192, lr_0 = 6.5000e-04
Validation auc = 0.474178
Epoch 5
Validation auc = 0.512676
Epoch 6
Validation auc = 0.557746
Epoch 7
Validation auc = 0.588732
Epoch 8
Validation auc = 0.604695
Epoch 9
Loss = 6.3120e-01, PNorm = 32.9028, GNorm = 1.4718, lr_0 = 9.9743e-04
Validation auc = 0.591080
Epoch 10
Validation auc = 0.595775
Epoch 11
Validation auc = 0.619249
Epoch 12
Validation auc = 0.618310
Epoch 13
Validation auc = 0.617840
Epoch 14
Loss = 5.7334e-01, PNorm = 32.9941, GNorm = 1.3431, lr_0 = 9.8890e-04
Validation auc = 0.665728
Epoch 15
Validation auc = 0.670892
Epoch 16
Validation auc = 0.627700
Epoch 17
Validation auc = 0.648357
Epoch 18
Validation auc = 0.653991
Epoch 19
Loss = 5.3744e-01, PNorm = 33.0873, GNorm = 0.8824, lr_0 = 9.8045e-04
Validation auc = 0.636620
Epoch 20
Validation auc = 0.649296
Epoch 21
Validation auc = 0.684977
Epoch 22
Validation auc = 0.684038
Epoch 23
Validation auc = 0.655399
Epoch 24
Loss = 4.3475e-01, PNorm = 33.2045, GNorm = 1.2412, lr_0 = 9.7207e-04
Validation auc = 0.663850
Epoch 25
Validation auc = 0.672300
Epoch 26
Validation auc = 0.678404
Epoch 27
Validation auc = 0.718779
Epoch 28
Validation auc = 0.654930
Epoch 29
Loss = 4.6360e-01, PNorm = 33.3213, GNorm = 1.5861, lr_0 = 9.6376e-04
Validation auc = 0.645540
Epoch 30
Validation auc = 0.652113
Epoch 31
Validation auc = 0.659155
Epoch 32
Validation auc = 0.658685
Epoch 33
Validation auc = 0.669014
Epoch 34
Loss = 4.0443e-01, PNorm = 33.4517, GNorm = 0.9846, lr_0 = 9.5552e-04
Validation auc = 0.683568
Epoch 35
Validation auc = 0.714085
Epoch 36
Validation auc = 0.684977
Epoch 37
Validation auc = 0.686385
Epoch 38
Validation auc = 0.728169
Epoch 39
Loss = 4.2701e-01, PNorm = 33.6010, GNorm = 1.8633, lr_0 = 9.4735e-04
Validation auc = 0.695305
Epoch 40
Validation auc = 0.715023
Epoch 41
Validation auc = 0.716901
Epoch 42
Validation auc = 0.682629
Epoch 43
Validation auc = 0.681221
Epoch 44
Loss = 2.8680e-01, PNorm = 33.7643, GNorm = 1.6264, lr_0 = 9.3925e-04
Validation auc = 0.658216
Epoch 45
Validation auc = 0.680282
Epoch 46
Validation auc = 0.675117
Epoch 47
Validation auc = 0.722066
Epoch 48
Validation auc = 0.686385
Epoch 49
Loss = 2.8705e-01, PNorm = 33.9166, GNorm = 1.0504, lr_0 = 9.3122e-04
Validation auc = 0.697183
Epoch 50
Validation auc = 0.694366
Epoch 51
Validation auc = 0.682160
Epoch 52
Validation auc = 0.707042
Epoch 53
Validation auc = 0.680282
Epoch 54
Loss = 3.7223e-01, PNorm = 34.0643, GNorm = 4.3067, lr_0 = 9.2326e-04
Validation auc = 0.699531
Epoch 55
Validation auc = 0.722066
Epoch 56
Validation auc = 0.672770
Epoch 57
Validation auc = 0.670892
Epoch 58
Validation auc = 0.682160
Epoch 59
Loss = 3.1812e-01, PNorm = 34.2037, GNorm = 1.8342, lr_0 = 9.1537e-04
Validation auc = 0.675587
Epoch 60
Validation auc = 0.675587
Epoch 61
Validation auc = 0.704225
Epoch 62
Validation auc = 0.702817
Epoch 63
Validation auc = 0.696714
Epoch 64
Loss = 3.0228e-01, PNorm = 34.3537, GNorm = 1.7831, lr_0 = 9.0754e-04
Validation auc = 0.711268
Epoch 65
Validation auc = 0.706573
Epoch 66
Validation auc = 0.691549
Epoch 67
Validation auc = 0.703756
Epoch 68
Validation auc = 0.706103
Epoch 69
Loss = 1.5646e-01, PNorm = 34.4958, GNorm = 0.7650, lr_0 = 8.9978e-04
Validation auc = 0.700469
Epoch 70
Validation auc = 0.711737
Epoch 71
Validation auc = 0.723944
Epoch 72
Validation auc = 0.706573
Epoch 73
Validation auc = 0.699531
Epoch 74
Loss = 1.5240e-01, PNorm = 34.6394, GNorm = 2.1398, lr_0 = 8.9209e-04
Validation auc = 0.696244
Epoch 75
Validation auc = 0.718310
Epoch 76
Validation auc = 0.718779
Epoch 77
Validation auc = 0.712207
Epoch 78
Validation auc = 0.693897
Epoch 79
Loss = 1.4831e-01, PNorm = 34.7824, GNorm = 2.2491, lr_0 = 8.8447e-04
Validation auc = 0.700939
Epoch 80
Validation auc = 0.694366
Epoch 81
Validation auc = 0.707042
Epoch 82
Validation auc = 0.677934
Epoch 83
Validation auc = 0.692488
Epoch 84
Loss = 1.6080e-01, PNorm = 34.9011, GNorm = 1.6740, lr_0 = 8.7691e-04
Validation auc = 0.675587
Epoch 85
Validation auc = 0.708451
Epoch 86
Validation auc = 0.723474
Epoch 87
Validation auc = 0.706573
Epoch 88
Validation auc = 0.673709
Epoch 89
Loss = 1.7373e-01, PNorm = 35.0174, GNorm = 1.2989, lr_0 = 8.6941e-04
Validation auc = 0.683099
Epoch 90
Validation auc = 0.672770
Epoch 91
Validation auc = 0.689671
Epoch 92
Validation auc = 0.715023
Epoch 93
Validation auc = 0.696244
Epoch 94
Loss = 1.8543e-01, PNorm = 35.1286, GNorm = 2.9652, lr_0 = 8.6198e-04
Validation auc = 0.697653
Epoch 95
Validation auc = 0.707512
Epoch 96
Validation auc = 0.677465
Epoch 97
Validation auc = 0.692488
Epoch 98
Validation auc = 0.696714
Epoch 99
Loss = 1.6722e-01, PNorm = 35.2411, GNorm = 1.0553, lr_0 = 8.5461e-04
Validation auc = 0.653052
Epoch 100
Validation auc = 0.658685
Epoch 101
Validation auc = 0.683568
Epoch 102
Validation auc = 0.659155
Epoch 103
Validation auc = 0.667136
Epoch 104
Loss = 5.0178e-02, PNorm = 35.3598, GNorm = 0.6361, lr_0 = 8.4730e-04
Validation auc = 0.683099
Epoch 105
Validation auc = 0.692488
Epoch 106
Validation auc = 0.692958
Epoch 107
Validation auc = 0.702817
Epoch 108
Validation auc = 0.713146
Epoch 109
Loss = 1.0169e-01, PNorm = 35.4691, GNorm = 0.9616, lr_0 = 8.4006e-04
Validation auc = 0.703286
Epoch 110
Validation auc = 0.707042
Epoch 111
Validation auc = 0.669953
Epoch 112
Validation auc = 0.689202
Epoch 113
Validation auc = 0.691080
Epoch 114
Loss = 6.3296e-02, PNorm = 35.5637, GNorm = 0.6811, lr_0 = 8.3288e-04
Validation auc = 0.673239
Epoch 115
Validation auc = 0.680751
Epoch 116
Validation auc = 0.689671
Epoch 117
Validation auc = 0.671362
Epoch 118
Validation auc = 0.673239
Epoch 119
Loss = 5.8338e-02, PNorm = 35.6464, GNorm = 0.9615, lr_0 = 8.2576e-04
Validation auc = 0.680751
Epoch 120
Validation auc = 0.665258
Epoch 121
Validation auc = 0.684038
Epoch 122
Validation auc = 0.690141
Epoch 123
Validation auc = 0.693897
Epoch 124
Loss = 1.3776e-01, PNorm = 35.7228, GNorm = 2.4589, lr_0 = 8.1870e-04
Validation auc = 0.713146
Epoch 125
Validation auc = 0.684038
Epoch 126
Validation auc = 0.678873
Epoch 127
Validation auc = 0.683099
Epoch 128
Validation auc = 0.672300
Epoch 129
Loss = 1.2264e-01, PNorm = 35.7982, GNorm = 3.5747, lr_0 = 8.1170e-04
Validation auc = 0.688732
Epoch 130
Validation auc = 0.709390
Epoch 131
Validation auc = 0.676995
Epoch 132
Validation auc = 0.698122
Epoch 133
Validation auc = 0.707042
Epoch 134
Loss = 8.3445e-02, PNorm = 35.8753, GNorm = 0.4174, lr_0 = 8.0476e-04
Validation auc = 0.662911
Epoch 135
Validation auc = 0.668545
Epoch 136
Validation auc = 0.695305
Epoch 137
Validation auc = 0.691080
Epoch 138
Validation auc = 0.662441
Epoch 139
Loss = 1.5876e-01, PNorm = 35.9558, GNorm = 1.3479, lr_0 = 7.9788e-04
Validation auc = 0.655869
Epoch 140
Validation auc = 0.662441
Epoch 141
Validation auc = 0.676526
Epoch 142
Validation auc = 0.683099
Epoch 143
Validation auc = 0.691549
Epoch 144
Loss = 5.6064e-02, PNorm = 36.0390, GNorm = 0.6457, lr_0 = 7.9106e-04
Validation auc = 0.687324
Epoch 145
Validation auc = 0.683568
Epoch 146
Validation auc = 0.682629
Epoch 147
Validation auc = 0.681221
Epoch 148
Validation auc = 0.661502
Epoch 149
Loss = 5.9488e-02, PNorm = 36.1179, GNorm = 1.4816, lr_0 = 7.8430e-04
Validation auc = 0.659624
Epoch 150
Validation auc = 0.689202
Epoch 151
Validation auc = 0.678404
Epoch 152
Validation auc = 0.668075
Epoch 153
Validation auc = 0.690141
Epoch 154
Loss = 1.1871e-01, PNorm = 36.1815, GNorm = 5.0092, lr_0 = 7.7759e-04
Validation auc = 0.692958
Epoch 155
Validation auc = 0.686385
Epoch 156
Validation auc = 0.698122
Epoch 157
Validation auc = 0.693427
Epoch 158
Validation auc = 0.684038
Epoch 159
Loss = 1.0279e-01, PNorm = 36.2437, GNorm = 2.2143, lr_0 = 7.7095e-04
Validation auc = 0.691549
Epoch 160
Validation auc = 0.701878
Epoch 161
Validation auc = 0.694366
Epoch 162
Validation auc = 0.680282
Epoch 163
Validation auc = 0.674648
Epoch 164
Loss = 3.8353e-02, PNorm = 36.3039, GNorm = 0.5476, lr_0 = 7.6436e-04
Validation auc = 0.681221
Epoch 165
Validation auc = 0.687793
Epoch 166
Validation auc = 0.689671
Epoch 167
Validation auc = 0.686385
Epoch 168
Validation auc = 0.688732
Epoch 169
Loss = 2.9647e-02, PNorm = 36.3612, GNorm = 0.2976, lr_0 = 7.5782e-04
Validation auc = 0.695305
Epoch 170
Validation auc = 0.697653
Epoch 171
Validation auc = 0.694836
Epoch 172
Validation auc = 0.698592
Epoch 173
Validation auc = 0.690141
Epoch 174
Loss = 3.1156e-02, PNorm = 36.4114, GNorm = 1.0154, lr_0 = 7.5134e-04
Validation auc = 0.683568
Epoch 175
Validation auc = 0.688732
Epoch 176
Validation auc = 0.682160
Epoch 177
Validation auc = 0.679812
Epoch 178
Validation auc = 0.695305
Epoch 179
Loss = 3.2689e-02, PNorm = 36.4638, GNorm = 0.9210, lr_0 = 7.4492e-04
Validation auc = 0.698122
Epoch 180
Validation auc = 0.690141
Epoch 181
Validation auc = 0.690141
Epoch 182
Validation auc = 0.697183
Epoch 183
Validation auc = 0.690610
Epoch 184
Loss = 1.8594e-02, PNorm = 36.5129, GNorm = 0.2396, lr_0 = 7.3855e-04
Validation auc = 0.678873
Epoch 185
Validation auc = 0.684507
Epoch 186
Validation auc = 0.683099
Epoch 187
Validation auc = 0.684507
Epoch 188
Validation auc = 0.683099
Epoch 189
Loss = 1.3155e-02, PNorm = 36.5561, GNorm = 0.6330, lr_0 = 7.3224e-04
Validation auc = 0.687793
Epoch 190
Validation auc = 0.694836
Epoch 191
Validation auc = 0.697653
Epoch 192
Validation auc = 0.692488
Epoch 193
Validation auc = 0.697653
Epoch 194
Loss = 2.1191e-02, PNorm = 36.5935, GNorm = 0.6371, lr_0 = 7.2598e-04
Validation auc = 0.689202
Epoch 195
Validation auc = 0.678404
Epoch 196
Validation auc = 0.687793
Epoch 197
Validation auc = 0.683099
Epoch 198
Validation auc = 0.686385
Epoch 199
Loss = 8.4346e-03, PNorm = 36.6346, GNorm = 0.1693, lr_0 = 7.1977e-04
Validation auc = 0.682629
Epoch 200
Validation auc = 0.685915
Epoch 201
Validation auc = 0.696714
Epoch 202
Validation auc = 0.700939
Epoch 203
Validation auc = 0.691549
Epoch 204
Loss = 2.3228e-02, PNorm = 36.6750, GNorm = 0.2104, lr_0 = 7.1362e-04
Validation auc = 0.695305
Epoch 205
Validation auc = 0.692019
Epoch 206
Validation auc = 0.686854
Epoch 207
Validation auc = 0.675587
Epoch 208
Validation auc = 0.679812
Epoch 209
Loss = 7.5514e-03, PNorm = 36.7116, GNorm = 0.2261, lr_0 = 7.0752e-04
Validation auc = 0.680751
Epoch 210
Validation auc = 0.689202
Epoch 211
Validation auc = 0.689202
Epoch 212
Validation auc = 0.684507
Epoch 213
Validation auc = 0.687793
Epoch 214
Loss = 3.0585e-02, PNorm = 36.7502, GNorm = 1.0465, lr_0 = 7.0147e-04
Validation auc = 0.699531
Epoch 215
Validation auc = 0.679812
Epoch 216
Validation auc = 0.662441
Epoch 217
Validation auc = 0.679343
Epoch 218
Validation auc = 0.677465
Epoch 219
Loss = 1.7684e-02, PNorm = 36.7885, GNorm = 0.4235, lr_0 = 6.9548e-04
Validation auc = 0.653991
Epoch 220
Validation auc = 0.651174
Epoch 221
Validation auc = 0.701408
Epoch 222
Validation auc = 0.688263
Epoch 223
Validation auc = 0.678404
Epoch 224
Loss = 1.2507e-02, PNorm = 36.8447, GNorm = 0.3382, lr_0 = 6.8953e-04
Validation auc = 0.681221
Epoch 225
Validation auc = 0.689202
Epoch 226
Validation auc = 0.686385
Epoch 227
Validation auc = 0.685915
Epoch 228
Validation auc = 0.683568
Epoch 229
Loss = 7.6557e-03, PNorm = 36.8979, GNorm = 0.2296, lr_0 = 6.8364e-04
Validation auc = 0.681221
Epoch 230
Validation auc = 0.678873
Epoch 231
Validation auc = 0.669014
Epoch 232
Validation auc = 0.667136
Epoch 233
Validation auc = 0.655399
Epoch 234
Loss = 5.1884e-02, PNorm = 36.9458, GNorm = 2.1429, lr_0 = 6.7779e-04
Validation auc = 0.664789
Epoch 235
Validation auc = 0.680282
Epoch 236
Validation auc = 0.681221
Epoch 237
Validation auc = 0.680282
Epoch 238
Validation auc = 0.690141
Epoch 239
Loss = 2.4529e-02, PNorm = 36.9940, GNorm = 0.1863, lr_0 = 6.7200e-04
Validation auc = 0.698122
Epoch 240
Validation auc = 0.700000
Epoch 241
Validation auc = 0.683099
Epoch 242
Validation auc = 0.676995
Epoch 243
Validation auc = 0.681221
Epoch 244
Loss = 1.5333e-02, PNorm = 37.0285, GNorm = 0.7671, lr_0 = 6.6625e-04
Validation auc = 0.686854
Epoch 245
Validation auc = 0.689671
Epoch 246
Validation auc = 0.691549
Epoch 247
Validation auc = 0.689202
Epoch 248
Validation auc = 0.684507
Epoch 249
Loss = 3.6855e-03, PNorm = 37.0591, GNorm = 0.1051, lr_0 = 6.6056e-04
Validation auc = 0.682629
Epoch 250
Validation auc = 0.685446
Epoch 251
Validation auc = 0.691549
Epoch 252
Validation auc = 0.682160
Epoch 253
Validation auc = 0.676995
Epoch 254
Loss = 1.2030e-02, PNorm = 37.0938, GNorm = 0.6773, lr_0 = 6.5491e-04
Validation auc = 0.678873
Epoch 255
Validation auc = 0.691549
Epoch 256
Validation auc = 0.692958
Epoch 257
Validation auc = 0.677934
Epoch 258
Validation auc = 0.666197
Epoch 259
Loss = 5.8090e-03, PNorm = 37.1307, GNorm = 0.2146, lr_0 = 6.4931e-04
Validation auc = 0.657746
Epoch 260
Validation auc = 0.651643
Epoch 261
Validation auc = 0.662911
Epoch 262
Validation auc = 0.675117
Epoch 263
Validation auc = 0.676526
Epoch 264
Loss = 3.2887e-03, PNorm = 37.1661, GNorm = 0.0657, lr_0 = 6.4376e-04
Validation auc = 0.674648
Epoch 265
Validation auc = 0.679343
Epoch 266
Validation auc = 0.685446
Epoch 267
Validation auc = 0.683568
Epoch 268
Validation auc = 0.668545
Epoch 269
Loss = 8.1347e-03, PNorm = 37.1987, GNorm = 0.1198, lr_0 = 6.3826e-04
Validation auc = 0.660563
Epoch 270
Validation auc = 0.662441
Epoch 271
Validation auc = 0.665258
Epoch 272
Validation auc = 0.670892
Epoch 273
Validation auc = 0.669014
Epoch 274
Loss = 3.6600e-02, PNorm = 37.2276, GNorm = 1.5558, lr_0 = 6.3280e-04
Validation auc = 0.675587
Epoch 275
Validation auc = 0.679343
Epoch 276
Validation auc = 0.681221
Epoch 277
Validation auc = 0.684977
Epoch 278
Validation auc = 0.689671
Epoch 279
Loss = 1.0573e-02, PNorm = 37.2550, GNorm = 0.6196, lr_0 = 6.2739e-04
Validation auc = 0.687324
Epoch 280
Validation auc = 0.684977
Epoch 281
Validation auc = 0.686854
Epoch 282
Validation auc = 0.687793
Epoch 283
Validation auc = 0.684977
Epoch 284
Loss = 6.8882e-03, PNorm = 37.2782, GNorm = 0.2519, lr_0 = 6.2203e-04
Validation auc = 0.676526
Epoch 285
Validation auc = 0.673239
Epoch 286
Validation auc = 0.675587
Epoch 287
Validation auc = 0.676526
Epoch 288
Validation auc = 0.676995
Epoch 289
Loss = 7.6256e-03, PNorm = 37.3011, GNorm = 0.4387, lr_0 = 6.1671e-04
Validation auc = 0.673709
Epoch 290
Validation auc = 0.668545
Epoch 291
Validation auc = 0.668545
Epoch 292
Validation auc = 0.675117
Epoch 293
Validation auc = 0.681221
Epoch 294
Loss = 9.7579e-03, PNorm = 37.3262, GNorm = 0.2042, lr_0 = 6.1144e-04
Validation auc = 0.679343
Epoch 295
Validation auc = 0.680751
Epoch 296
Validation auc = 0.678404
Epoch 297
Validation auc = 0.679343
Epoch 298
Validation auc = 0.683568
Epoch 299
Loss = 1.1605e-02, PNorm = 37.3494, GNorm = 0.3738, lr_0 = 6.0621e-04
Validation auc = 0.681690
Model 0 best validation auc = 0.728169 on epoch 38
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.916918
Ensemble test auc = 0.916918
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=400, bias=False)
        (W_h): Linear(in_features=400, out_features=400, bias=False)
        (W_o): Linear(in_features=533, out_features=400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=628, out_features=1, bias=True)
  )
)
Number of parameters = 433,029
Moving model to cuda
Epoch 0
Validation auc = 0.394366
Epoch 1
Validation auc = 0.407981
Epoch 2
Validation auc = 0.430047
Epoch 3
Validation auc = 0.451174
Epoch 4
Loss = 7.6991e-01, PNorm = 32.8312, GNorm = 1.7869, lr_0 = 6.5000e-04
Validation auc = 0.468545
Epoch 5
Validation auc = 0.517840
Epoch 6
Validation auc = 0.553052
Epoch 7
Validation auc = 0.580282
Epoch 8
Validation auc = 0.590610
Epoch 9
Loss = 5.7584e-01, PNorm = 32.9082, GNorm = 2.6214, lr_0 = 9.9743e-04
Validation auc = 0.608451
Epoch 10
Validation auc = 0.641315
Epoch 11
Validation auc = 0.613615
Epoch 12
Validation auc = 0.594366
Epoch 13
Validation auc = 0.609390
Epoch 14
Loss = 4.9867e-01, PNorm = 32.9884, GNorm = 1.5633, lr_0 = 9.8890e-04
Validation auc = 0.647418
Epoch 15
Validation auc = 0.662441
Epoch 16
Validation auc = 0.662911
Epoch 17
Validation auc = 0.683099
Epoch 18
Validation auc = 0.692958
Epoch 19
Loss = 4.7397e-01, PNorm = 33.0740, GNorm = 0.8621, lr_0 = 9.8045e-04
Validation auc = 0.648826
Epoch 20
Validation auc = 0.650704
Epoch 21
Validation auc = 0.653052
Epoch 22
Validation auc = 0.655869
Epoch 23
Validation auc = 0.686854
Epoch 24
Loss = 4.1060e-01, PNorm = 33.1729, GNorm = 1.1643, lr_0 = 9.7207e-04
Validation auc = 0.685915
Epoch 25
Validation auc = 0.715493
Epoch 26
Validation auc = 0.709390
Epoch 27
Validation auc = 0.707512
Epoch 28
Validation auc = 0.723005
Epoch 29
Loss = 4.3089e-01, PNorm = 33.2950, GNorm = 0.7694, lr_0 = 9.6376e-04
Validation auc = 0.653521
Epoch 30
Validation auc = 0.676526
Epoch 31
Validation auc = 0.657746
Epoch 32
Validation auc = 0.642254
Epoch 33
Validation auc = 0.666667
Epoch 34
Loss = 4.3236e-01, PNorm = 33.4203, GNorm = 2.4498, lr_0 = 9.5552e-04
Validation auc = 0.670423
Epoch 35
Validation auc = 0.666667
Epoch 36
Validation auc = 0.689202
Epoch 37
Validation auc = 0.712207
Epoch 38
Validation auc = 0.664319
Epoch 39
Loss = 3.3038e-01, PNorm = 33.5620, GNorm = 1.4447, lr_0 = 9.4735e-04
Validation auc = 0.712676
Epoch 40
Validation auc = 0.689202
Epoch 41
Validation auc = 0.680282
Epoch 42
Validation auc = 0.721596
Epoch 43
Validation auc = 0.679343
Epoch 44
Loss = 3.4613e-01, PNorm = 33.7049, GNorm = 3.4318, lr_0 = 9.3925e-04
Validation auc = 0.663850
Epoch 45
Validation auc = 0.685915
Epoch 46
Validation auc = 0.680751
Epoch 47
Validation auc = 0.663380
Epoch 48
Validation auc = 0.685915
Epoch 49
Loss = 2.9317e-01, PNorm = 33.8547, GNorm = 1.1502, lr_0 = 9.3122e-04
Validation auc = 0.716432
Epoch 50
Validation auc = 0.702817
Epoch 51
Validation auc = 0.699531
Epoch 52
Validation auc = 0.700469
Epoch 53
Validation auc = 0.700939
Epoch 54
Loss = 2.9796e-01, PNorm = 34.0033, GNorm = 1.7757, lr_0 = 9.2326e-04
Validation auc = 0.705164
Epoch 55
Validation auc = 0.700939
Epoch 56
Validation auc = 0.686385
Epoch 57
Validation auc = 0.701878
Epoch 58
Validation auc = 0.654460
Epoch 59
Loss = 3.5021e-01, PNorm = 34.1584, GNorm = 1.1529, lr_0 = 9.1537e-04
Validation auc = 0.724413
Epoch 60
Validation auc = 0.659624
Epoch 61
Validation auc = 0.673239
Epoch 62
Validation auc = 0.680282
Epoch 63
Validation auc = 0.659155
Epoch 64
Loss = 2.3265e-01, PNorm = 34.2920, GNorm = 1.6384, lr_0 = 9.0754e-04
Validation auc = 0.669014
Epoch 65
Validation auc = 0.692488
Epoch 66
Validation auc = 0.682629
Epoch 67
Validation auc = 0.661972
Epoch 68
Validation auc = 0.697653
Epoch 69
Loss = 2.2876e-01, PNorm = 34.4405, GNorm = 1.4999, lr_0 = 8.9978e-04
Validation auc = 0.687793
Epoch 70
Validation auc = 0.686385
Epoch 71
Validation auc = 0.704695
Epoch 72
Validation auc = 0.691080
Epoch 73
Validation auc = 0.669953
Epoch 74
Loss = 2.1579e-01, PNorm = 34.5926, GNorm = 3.3227, lr_0 = 8.9209e-04
Validation auc = 0.666667
Epoch 75
Validation auc = 0.678873
Epoch 76
Validation auc = 0.704695
Epoch 77
Validation auc = 0.703286
Epoch 78
Validation auc = 0.707981
Epoch 79
Loss = 1.9410e-01, PNorm = 34.7334, GNorm = 1.0846, lr_0 = 8.8447e-04
Validation auc = 0.663380
Epoch 80
Validation auc = 0.667606
Epoch 81
Validation auc = 0.648826
Epoch 82
Validation auc = 0.634272
Epoch 83
Validation auc = 0.653052
Epoch 84
Loss = 2.0235e-01, PNorm = 34.8695, GNorm = 1.2249, lr_0 = 8.7691e-04
Validation auc = 0.682160
Epoch 85
Validation auc = 0.663850
Epoch 86
Validation auc = 0.671831
Epoch 87
Validation auc = 0.687793
Epoch 88
Validation auc = 0.662911
Epoch 89
Loss = 1.0921e-01, PNorm = 34.9972, GNorm = 0.5878, lr_0 = 8.6941e-04
Validation auc = 0.671831
Epoch 90
Validation auc = 0.682160
Epoch 91
Validation auc = 0.693427
Epoch 92
Validation auc = 0.683568
Epoch 93
Validation auc = 0.680282
Epoch 94
Loss = 1.0819e-01, PNorm = 35.1151, GNorm = 1.0339, lr_0 = 8.6198e-04
Validation auc = 0.698592
Epoch 95
Validation auc = 0.663850
Epoch 96
Validation auc = 0.690610
Epoch 97
Validation auc = 0.688732
Epoch 98
Validation auc = 0.664319
Epoch 99
Loss = 1.1105e-01, PNorm = 35.2173, GNorm = 2.2471, lr_0 = 8.5461e-04
Validation auc = 0.679343
Epoch 100
Validation auc = 0.672770
Epoch 101
Validation auc = 0.652582
Epoch 102
Validation auc = 0.680751
Epoch 103
Validation auc = 0.658216
Epoch 104
Loss = 1.0221e-01, PNorm = 35.3207, GNorm = 1.2215, lr_0 = 8.4730e-04
Validation auc = 0.661033
Epoch 105
Validation auc = 0.677465
Epoch 106
Validation auc = 0.679812
Epoch 107
Validation auc = 0.685446
Epoch 108
Validation auc = 0.700939
Epoch 109
Loss = 7.4233e-02, PNorm = 35.4190, GNorm = 2.4137, lr_0 = 8.4006e-04
Validation auc = 0.667136
Epoch 110
Validation auc = 0.658685
Epoch 111
Validation auc = 0.698122
Epoch 112
Validation auc = 0.675117
Epoch 113
Validation auc = 0.662441
Epoch 114
Loss = 1.6239e-01, PNorm = 35.5244, GNorm = 1.4233, lr_0 = 8.3288e-04
Validation auc = 0.700000
Epoch 115
Validation auc = 0.680751
Epoch 116
Validation auc = 0.648826
Epoch 117
Validation auc = 0.633803
Epoch 118
Validation auc = 0.661033
Epoch 119
Loss = 1.0336e-01, PNorm = 35.6216, GNorm = 3.3179, lr_0 = 8.2576e-04
Validation auc = 0.651174
Epoch 120
Validation auc = 0.643192
Epoch 121
Validation auc = 0.717371
Epoch 122
Validation auc = 0.663850
Epoch 123
Validation auc = 0.664319
Epoch 124
Loss = 1.2336e-01, PNorm = 35.7242, GNorm = 1.0855, lr_0 = 8.1870e-04
Validation auc = 0.692958
Epoch 125
Validation auc = 0.662441
Epoch 126
Validation auc = 0.641784
Epoch 127
Validation auc = 0.673239
Epoch 128
Validation auc = 0.704225
Epoch 129
Loss = 1.7971e-01, PNorm = 35.8349, GNorm = 1.7154, lr_0 = 8.1170e-04
Validation auc = 0.670423
Epoch 130
Validation auc = 0.655399
Epoch 131
Validation auc = 0.667606
Epoch 132
Validation auc = 0.692019
Epoch 133
Validation auc = 0.673239
Epoch 134
Loss = 1.1796e-01, PNorm = 35.9337, GNorm = 0.8746, lr_0 = 8.0476e-04
Validation auc = 0.657277
Epoch 135
Validation auc = 0.659155
Epoch 136
Validation auc = 0.669484
Epoch 137
Validation auc = 0.672770
Epoch 138
Validation auc = 0.670423
Epoch 139
Loss = 1.2317e-01, PNorm = 36.0177, GNorm = 0.2925, lr_0 = 7.9788e-04
Validation auc = 0.673709
Epoch 140
Validation auc = 0.674648
Epoch 141
Validation auc = 0.678404
Epoch 142
Validation auc = 0.680751
Epoch 143
Validation auc = 0.682160
Epoch 144
Loss = 7.4755e-02, PNorm = 36.0816, GNorm = 0.7856, lr_0 = 7.9106e-04
Validation auc = 0.681690
Epoch 145
Validation auc = 0.687324
Epoch 146
Validation auc = 0.693427
Epoch 147
Validation auc = 0.673709
Epoch 148
Validation auc = 0.665258
Epoch 149
Loss = 9.3587e-02, PNorm = 36.1385, GNorm = 0.3590, lr_0 = 7.8430e-04
Validation auc = 0.677465
Epoch 150
Validation auc = 0.677934
Epoch 151
Validation auc = 0.660094
Epoch 152
Validation auc = 0.654930
Epoch 153
Validation auc = 0.663850
Epoch 154
Loss = 4.3258e-02, PNorm = 36.1928, GNorm = 0.5216, lr_0 = 7.7759e-04
Validation auc = 0.669484
Epoch 155
Validation auc = 0.672770
Epoch 156
Validation auc = 0.660094
Epoch 157
Validation auc = 0.659624
Epoch 158
Validation auc = 0.672770
Epoch 159
Loss = 6.2072e-02, PNorm = 36.2523, GNorm = 1.3371, lr_0 = 7.7095e-04
Validation auc = 0.676995
Epoch 160
Validation auc = 0.679343
Epoch 161
Validation auc = 0.689671
Epoch 162
Validation auc = 0.687793
Epoch 163
Validation auc = 0.667606
Epoch 164
Loss = 2.6629e-02, PNorm = 36.3145, GNorm = 0.8033, lr_0 = 7.6436e-04
Validation auc = 0.654460
Epoch 165
Validation auc = 0.671362
Epoch 166
Validation auc = 0.668075
Epoch 167
Validation auc = 0.643192
Epoch 168
Validation auc = 0.662441
Epoch 169
Loss = 1.0012e-01, PNorm = 36.3811, GNorm = 0.5861, lr_0 = 7.5782e-04
Validation auc = 0.700469
Epoch 170
Validation auc = 0.696714
Epoch 171
Validation auc = 0.678873
Epoch 172
Validation auc = 0.665258
Epoch 173
Validation auc = 0.668545
Epoch 174
Loss = 8.7606e-02, PNorm = 36.4432, GNorm = 1.0596, lr_0 = 7.5134e-04
Validation auc = 0.676995
Epoch 175
Validation auc = 0.681690
Epoch 176
Validation auc = 0.667136
Epoch 177
Validation auc = 0.653991
Epoch 178
Validation auc = 0.659624
Epoch 179
Loss = 3.0833e-02, PNorm = 36.5083, GNorm = 0.3401, lr_0 = 7.4492e-04
Validation auc = 0.676056
Epoch 180
Validation auc = 0.680282
Epoch 181
Validation auc = 0.668075
Epoch 182
Validation auc = 0.663850
Epoch 183
Validation auc = 0.670423
Epoch 184
Loss = 1.9741e-02, PNorm = 36.5700, GNorm = 0.2180, lr_0 = 7.3855e-04
Validation auc = 0.676995
Epoch 185
Validation auc = 0.682629
Epoch 186
Validation auc = 0.681690
Epoch 187
Validation auc = 0.674178
Epoch 188
Validation auc = 0.674648
Epoch 189
Loss = 1.7941e-02, PNorm = 36.6206, GNorm = 0.5888, lr_0 = 7.3224e-04
Validation auc = 0.673709
Epoch 190
Validation auc = 0.674178
Epoch 191
Validation auc = 0.676995
Epoch 192
Validation auc = 0.682160
Epoch 193
Validation auc = 0.680751
Epoch 194
Loss = 2.6403e-02, PNorm = 36.6692, GNorm = 0.7329, lr_0 = 7.2598e-04
Validation auc = 0.677934
Epoch 195
Validation auc = 0.674178
Epoch 196
Validation auc = 0.676995
Epoch 197
Validation auc = 0.680282
Epoch 198
Validation auc = 0.678404
Epoch 199
Loss = 2.6220e-02, PNorm = 36.7087, GNorm = 1.1583, lr_0 = 7.1977e-04
Validation auc = 0.672300
Epoch 200
Validation auc = 0.670892
Epoch 201
Validation auc = 0.667136
Epoch 202
Validation auc = 0.675587
Epoch 203
Validation auc = 0.674648
Epoch 204
Loss = 1.7934e-02, PNorm = 36.7436, GNorm = 0.7466, lr_0 = 7.1362e-04
Validation auc = 0.669014
Epoch 205
Validation auc = 0.675117
Epoch 206
Validation auc = 0.678404
Epoch 207
Validation auc = 0.680751
Epoch 208
Validation auc = 0.676526
Epoch 209
Loss = 2.6823e-02, PNorm = 36.7778, GNorm = 0.5019, lr_0 = 7.0752e-04
Validation auc = 0.683099
Epoch 210
Validation auc = 0.678404
Epoch 211
Validation auc = 0.671362
Epoch 212
Validation auc = 0.667606
Epoch 213
Validation auc = 0.668075
Epoch 214
Loss = 8.8836e-02, PNorm = 36.8136, GNorm = 0.7998, lr_0 = 7.0147e-04
Validation auc = 0.676526
Epoch 215
Validation auc = 0.667136
Epoch 216
Validation auc = 0.668545
Epoch 217
Validation auc = 0.686854
Epoch 218
Validation auc = 0.664319
Epoch 219
Loss = 9.2057e-03, PNorm = 36.8458, GNorm = 0.3649, lr_0 = 6.9548e-04
Validation auc = 0.649296
Epoch 220
Validation auc = 0.671831
Epoch 221
Validation auc = 0.692019
Epoch 222
Validation auc = 0.674648
Epoch 223
Validation auc = 0.666667
Epoch 224
Loss = 1.0697e-01, PNorm = 36.8926, GNorm = 3.1358, lr_0 = 6.8953e-04
Validation auc = 0.681221
Epoch 225
Validation auc = 0.693897
Epoch 226
Validation auc = 0.678873
Epoch 227
Validation auc = 0.674178
Epoch 228
Validation auc = 0.672300
Epoch 229
Loss = 5.2722e-02, PNorm = 36.9479, GNorm = 1.5367, lr_0 = 6.8364e-04
Validation auc = 0.668545
Epoch 230
Validation auc = 0.686854
Epoch 231
Validation auc = 0.683568
Epoch 232
Validation auc = 0.669953
Epoch 233
Validation auc = 0.671362
Epoch 234
Loss = 5.2923e-02, PNorm = 37.0213, GNorm = 0.8841, lr_0 = 6.7779e-04
Validation auc = 0.681221
Epoch 235
Validation auc = 0.684507
Epoch 236
Validation auc = 0.676056
Epoch 237
Validation auc = 0.683099
Epoch 238
Validation auc = 0.700939
Epoch 239
Loss = 4.6408e-02, PNorm = 37.0811, GNorm = 1.0204, lr_0 = 6.7200e-04
Validation auc = 0.702347
Epoch 240
Validation auc = 0.694366
Epoch 241
Validation auc = 0.690141
Epoch 242
Validation auc = 0.691080
Epoch 243
Validation auc = 0.689202
Epoch 244
Loss = 5.4767e-02, PNorm = 37.1257, GNorm = 0.8226, lr_0 = 6.6625e-04
Validation auc = 0.690141
Epoch 245
Validation auc = 0.682160
Epoch 246
Validation auc = 0.675587
Epoch 247
Validation auc = 0.673239
Epoch 248
Validation auc = 0.683099
Epoch 249
Loss = 3.6234e-02, PNorm = 37.1648, GNorm = 1.6405, lr_0 = 6.6056e-04
Validation auc = 0.689671
Epoch 250
Validation auc = 0.679812
Epoch 251
Validation auc = 0.680282
Epoch 252
Validation auc = 0.697183
Epoch 253
Validation auc = 0.713615
Epoch 254
Loss = 3.1455e-02, PNorm = 37.2075, GNorm = 1.1122, lr_0 = 6.5491e-04
Validation auc = 0.705164
Epoch 255
Validation auc = 0.685915
Epoch 256
Validation auc = 0.693427
Epoch 257
Validation auc = 0.702817
Epoch 258
Validation auc = 0.690141
Epoch 259
Loss = 1.4005e-02, PNorm = 37.2454, GNorm = 0.3711, lr_0 = 6.4931e-04
Validation auc = 0.668545
Epoch 260
Validation auc = 0.661033
Epoch 261
Validation auc = 0.676056
Epoch 262
Validation auc = 0.688263
Epoch 263
Validation auc = 0.686854
Epoch 264
Loss = 2.3340e-02, PNorm = 37.2892, GNorm = 0.5481, lr_0 = 6.4376e-04
Validation auc = 0.678404
Epoch 265
Validation auc = 0.677934
Epoch 266
Validation auc = 0.690141
Epoch 267
Validation auc = 0.693897
Epoch 268
Validation auc = 0.690141
Epoch 269
Loss = 2.3263e-02, PNorm = 37.3262, GNorm = 0.2506, lr_0 = 6.3826e-04
Validation auc = 0.683568
Epoch 270
Validation auc = 0.680751
Epoch 271
Validation auc = 0.681221
Epoch 272
Validation auc = 0.675117
Epoch 273
Validation auc = 0.669014
Epoch 274
Loss = 9.7488e-03, PNorm = 37.3541, GNorm = 0.1410, lr_0 = 6.3280e-04
Validation auc = 0.667136
Epoch 275
Validation auc = 0.663850
Epoch 276
Validation auc = 0.669953
Epoch 277
Validation auc = 0.676056
Epoch 278
Validation auc = 0.673239
Epoch 279
Loss = 7.9596e-03, PNorm = 37.3785, GNorm = 0.3043, lr_0 = 6.2739e-04
Validation auc = 0.664789
Epoch 280
Validation auc = 0.661972
Epoch 281
Validation auc = 0.682629
Epoch 282
Validation auc = 0.679343
Epoch 283
Validation auc = 0.664319
Epoch 284
Loss = 2.5178e-02, PNorm = 37.4071, GNorm = 1.8984, lr_0 = 6.2203e-04
Validation auc = 0.664789
Epoch 285
Validation auc = 0.676056
Epoch 286
Validation auc = 0.688732
Epoch 287
Validation auc = 0.696244
Epoch 288
Validation auc = 0.690141
Epoch 289
Loss = 1.1808e-02, PNorm = 37.4407, GNorm = 0.1222, lr_0 = 6.1671e-04
Validation auc = 0.680282
Epoch 290
Validation auc = 0.674178
Epoch 291
Validation auc = 0.676526
Epoch 292
Validation auc = 0.683568
Epoch 293
Validation auc = 0.687324
Epoch 294
Loss = 2.1944e-02, PNorm = 37.4734, GNorm = 1.3819, lr_0 = 6.1144e-04
Validation auc = 0.684038
Epoch 295
Validation auc = 0.672770
Epoch 296
Validation auc = 0.666197
Epoch 297
Validation auc = 0.662441
Epoch 298
Validation auc = 0.663380
Epoch 299
Loss = 2.7505e-02, PNorm = 37.5012, GNorm = 0.8860, lr_0 = 6.0621e-04
Validation auc = 0.670892
Model 0 best validation auc = 0.724413 on epoch 59
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.875376
Ensemble test auc = 0.875376
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=400, bias=False)
        (W_h): Linear(in_features=400, out_features=400, bias=False)
        (W_o): Linear(in_features=533, out_features=400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=628, out_features=1, bias=True)
  )
)
Number of parameters = 433,029
Moving model to cuda
Epoch 0
Validation auc = 0.398592
Epoch 1
Validation auc = 0.416432
Epoch 2
Validation auc = 0.445540
Epoch 3
Validation auc = 0.462441
Epoch 4
Loss = 7.2053e-01, PNorm = 32.8304, GNorm = 1.6783, lr_0 = 6.5000e-04
Validation auc = 0.476056
Epoch 5
Validation auc = 0.515962
Epoch 6
Validation auc = 0.572300
Epoch 7
Validation auc = 0.604695
Epoch 8
Validation auc = 0.599061
Epoch 9
Loss = 5.7336e-01, PNorm = 32.9338, GNorm = 2.6623, lr_0 = 9.9743e-04
Validation auc = 0.599531
Epoch 10
Validation auc = 0.655869
Epoch 11
Validation auc = 0.628638
Epoch 12
Validation auc = 0.656338
Epoch 13
Validation auc = 0.657277
Epoch 14
Loss = 4.7951e-01, PNorm = 33.0011, GNorm = 1.1217, lr_0 = 9.8890e-04
Validation auc = 0.620657
Epoch 15
Validation auc = 0.620657
Epoch 16
Validation auc = 0.633333
Epoch 17
Validation auc = 0.647418
Epoch 18
Validation auc = 0.631925
Epoch 19
Loss = 5.1966e-01, PNorm = 33.0693, GNorm = 1.4139, lr_0 = 9.8045e-04
Validation auc = 0.632864
Epoch 20
Validation auc = 0.646479
Epoch 21
Validation auc = 0.667136
Epoch 22
Validation auc = 0.666197
Epoch 23
Validation auc = 0.654460
Epoch 24
Loss = 4.8523e-01, PNorm = 33.1536, GNorm = 1.0870, lr_0 = 9.7207e-04
Validation auc = 0.648826
Epoch 25
Validation auc = 0.655399
Epoch 26
Validation auc = 0.663850
Epoch 27
Validation auc = 0.658216
Epoch 28
Validation auc = 0.670892
Epoch 29
Loss = 4.1552e-01, PNorm = 33.2578, GNorm = 1.1539, lr_0 = 9.6376e-04
Validation auc = 0.656808
Epoch 30
Validation auc = 0.666197
Epoch 31
Validation auc = 0.669953
Epoch 32
Validation auc = 0.693427
Epoch 33
Validation auc = 0.698122
Epoch 34
Loss = 3.9226e-01, PNorm = 33.3748, GNorm = 1.9182, lr_0 = 9.5552e-04
Validation auc = 0.674648
Epoch 35
Validation auc = 0.707042
Epoch 36
Validation auc = 0.658216
Epoch 37
Validation auc = 0.713146
Epoch 38
Validation auc = 0.723944
Epoch 39
Loss = 4.0503e-01, PNorm = 33.4849, GNorm = 0.8485, lr_0 = 9.4735e-04
Validation auc = 0.679812
Epoch 40
Validation auc = 0.653991
Epoch 41
Validation auc = 0.675117
Epoch 42
Validation auc = 0.669953
Epoch 43
Validation auc = 0.674648
Epoch 44
Loss = 3.5681e-01, PNorm = 33.6127, GNorm = 0.8210, lr_0 = 9.3925e-04
Validation auc = 0.707042
Epoch 45
Validation auc = 0.714554
Epoch 46
Validation auc = 0.692958
Epoch 47
Validation auc = 0.705164
Epoch 48
Validation auc = 0.695775
Epoch 49
Loss = 2.6962e-01, PNorm = 33.7476, GNorm = 0.7547, lr_0 = 9.3122e-04
Validation auc = 0.681221
Epoch 50
Validation auc = 0.711737
Epoch 51
Validation auc = 0.709859
Epoch 52
Validation auc = 0.667136
Epoch 53
Validation auc = 0.684977
Epoch 54
Loss = 2.9599e-01, PNorm = 33.8952, GNorm = 3.8818, lr_0 = 9.2326e-04
Validation auc = 0.697653
Epoch 55
Validation auc = 0.676526
Epoch 56
Validation auc = 0.706573
Epoch 57
Validation auc = 0.713146
Epoch 58
Validation auc = 0.685446
Epoch 59
Loss = 3.6308e-01, PNorm = 34.0351, GNorm = 1.8693, lr_0 = 9.1537e-04
Validation auc = 0.701408
Epoch 60
Validation auc = 0.672300
Epoch 61
Validation auc = 0.652113
Epoch 62
Validation auc = 0.682629
Epoch 63
Validation auc = 0.702817
Epoch 64
Loss = 2.0175e-01, PNorm = 34.1842, GNorm = 0.8097, lr_0 = 9.0754e-04
Validation auc = 0.669484
Epoch 65
Validation auc = 0.708920
Epoch 66
Validation auc = 0.696714
Epoch 67
Validation auc = 0.665258
Epoch 68
Validation auc = 0.675117
Epoch 69
Loss = 2.6886e-01, PNorm = 34.3337, GNorm = 4.7248, lr_0 = 8.9978e-04
Validation auc = 0.681221
Epoch 70
Validation auc = 0.674178
Epoch 71
Validation auc = 0.701878
Epoch 72
Validation auc = 0.694366
Epoch 73
Validation auc = 0.663850
Epoch 74
Loss = 1.5157e-01, PNorm = 34.4713, GNorm = 1.5148, lr_0 = 8.9209e-04
Validation auc = 0.658685
Epoch 75
Validation auc = 0.675587
Epoch 76
Validation auc = 0.665258
Epoch 77
Validation auc = 0.708451
Epoch 78
Validation auc = 0.700000
Epoch 79
Loss = 2.1220e-01, PNorm = 34.6089, GNorm = 3.9931, lr_0 = 8.8447e-04
Validation auc = 0.702817
Epoch 80
Validation auc = 0.705634
Epoch 81
Validation auc = 0.675587
Epoch 82
Validation auc = 0.671831
Epoch 83
Validation auc = 0.679812
Epoch 84
Loss = 1.4201e-01, PNorm = 34.7405, GNorm = 1.6790, lr_0 = 8.7691e-04
Validation auc = 0.688263
Epoch 85
Validation auc = 0.722535
Epoch 86
Validation auc = 0.700000
Epoch 87
Validation auc = 0.690610
Epoch 88
Validation auc = 0.713146
Epoch 89
Loss = 2.0386e-01, PNorm = 34.8548, GNorm = 2.8006, lr_0 = 8.6941e-04
Validation auc = 0.674648
Epoch 90
Validation auc = 0.654930
Epoch 91
Validation auc = 0.692488
Epoch 92
Validation auc = 0.702817
Epoch 93
Validation auc = 0.692488
Epoch 94
Loss = 2.2250e-01, PNorm = 34.9707, GNorm = 2.8089, lr_0 = 8.6198e-04
Validation auc = 0.676995
Epoch 95
Validation auc = 0.693427
Epoch 96
Validation auc = 0.685915
Epoch 97
Validation auc = 0.665728
Epoch 98
Validation auc = 0.676056
Epoch 99
Loss = 1.3956e-01, PNorm = 35.0976, GNorm = 0.8146, lr_0 = 8.5461e-04
Validation auc = 0.705634
Epoch 100
Validation auc = 0.695775
Epoch 101
Validation auc = 0.676056
Epoch 102
Validation auc = 0.677934
Epoch 103
Validation auc = 0.676056
Epoch 104
Loss = 1.0307e-01, PNorm = 35.2160, GNorm = 1.7097, lr_0 = 8.4730e-04
Validation auc = 0.682160
Epoch 105
Validation auc = 0.691080
Epoch 106
Validation auc = 0.672300
Epoch 107
Validation auc = 0.700939
Epoch 108
Validation auc = 0.673709
Epoch 109
Loss = 1.1222e-01, PNorm = 35.3305, GNorm = 2.9827, lr_0 = 8.4006e-04
Validation auc = 0.658216
Epoch 110
Validation auc = 0.684038
Epoch 111
Validation auc = 0.681221
Epoch 112
Validation auc = 0.657277
Epoch 113
Validation auc = 0.659155
Epoch 114
Loss = 2.7549e-01, PNorm = 35.4388, GNorm = 0.5607, lr_0 = 8.3288e-04
Validation auc = 0.685446
Epoch 115
Validation auc = 0.683099
Epoch 116
Validation auc = 0.660563
Epoch 117
Validation auc = 0.657277
Epoch 118
Validation auc = 0.671362
Epoch 119
Loss = 1.4588e-01, PNorm = 35.5479, GNorm = 1.2452, lr_0 = 8.2576e-04
Validation auc = 0.683568
Epoch 120
Validation auc = 0.679343
Epoch 121
Validation auc = 0.680282
Epoch 122
Validation auc = 0.689671
Epoch 123
Validation auc = 0.708920
Epoch 124
Loss = 1.5803e-01, PNorm = 35.6522, GNorm = 2.9545, lr_0 = 8.1870e-04
Validation auc = 0.676526
Epoch 125
Validation auc = 0.647887
Epoch 126
Validation auc = 0.662441
Epoch 127
Validation auc = 0.687324
Epoch 128
Validation auc = 0.674648
Epoch 129
Loss = 5.9285e-02, PNorm = 35.7497, GNorm = 0.9486, lr_0 = 8.1170e-04
Validation auc = 0.660563
Epoch 130
Validation auc = 0.669484
Epoch 131
Validation auc = 0.694366
Epoch 132
Validation auc = 0.698592
Epoch 133
Validation auc = 0.672300
Epoch 134
Loss = 6.8075e-02, PNorm = 35.8432, GNorm = 0.3624, lr_0 = 8.0476e-04
Validation auc = 0.650235
Epoch 135
Validation auc = 0.646479
Epoch 136
Validation auc = 0.650704
Epoch 137
Validation auc = 0.661502
Epoch 138
Validation auc = 0.669484
Epoch 139
Loss = 6.9531e-02, PNorm = 35.9368, GNorm = 0.9817, lr_0 = 7.9788e-04
Validation auc = 0.667606
Epoch 140
Validation auc = 0.679343
Epoch 141
Validation auc = 0.688263
Epoch 142
Validation auc = 0.704225
Epoch 143
Validation auc = 0.711268
Epoch 144
Loss = 8.9074e-02, PNorm = 36.0171, GNorm = 1.8164, lr_0 = 7.9106e-04
Validation auc = 0.682160
Epoch 145
Validation auc = 0.659155
Epoch 146
Validation auc = 0.659624
Epoch 147
Validation auc = 0.679812
Epoch 148
Validation auc = 0.682629
Epoch 149
Loss = 7.5236e-02, PNorm = 36.0979, GNorm = 0.6812, lr_0 = 7.8430e-04
Validation auc = 0.668075
Epoch 150
Validation auc = 0.659155
Epoch 151
Validation auc = 0.675117
Epoch 152
Validation auc = 0.678404
Epoch 153
Validation auc = 0.674648
Epoch 154
Loss = 6.6135e-02, PNorm = 36.1755, GNorm = 1.1678, lr_0 = 7.7759e-04
Validation auc = 0.668545
Epoch 155
Validation auc = 0.676526
Epoch 156
Validation auc = 0.688732
Epoch 157
Validation auc = 0.692488
Epoch 158
Validation auc = 0.664319
Epoch 159
Loss = 7.0979e-02, PNorm = 36.2489, GNorm = 1.6675, lr_0 = 7.7095e-04
Validation auc = 0.650704
Epoch 160
Validation auc = 0.658216
Epoch 161
Validation auc = 0.664319
Epoch 162
Validation auc = 0.652113
Epoch 163
Validation auc = 0.653521
Epoch 164
Loss = 4.3593e-02, PNorm = 36.3198, GNorm = 0.8623, lr_0 = 7.6436e-04
Validation auc = 0.663850
Epoch 165
Validation auc = 0.674178
Epoch 166
Validation auc = 0.679343
Epoch 167
Validation auc = 0.676056
Epoch 168
Validation auc = 0.672770
Epoch 169
Loss = 3.8793e-02, PNorm = 36.3869, GNorm = 0.7426, lr_0 = 7.5782e-04
Validation auc = 0.670423
Epoch 170
Validation auc = 0.670892
Epoch 171
Validation auc = 0.669014
Epoch 172
Validation auc = 0.665258
Epoch 173
Validation auc = 0.674178
Epoch 174
Loss = 2.6478e-02, PNorm = 36.4459, GNorm = 0.4348, lr_0 = 7.5134e-04
Validation auc = 0.679343
Epoch 175
Validation auc = 0.679812
Epoch 176
Validation auc = 0.662441
Epoch 177
Validation auc = 0.651643
Epoch 178
Validation auc = 0.657277
Epoch 179
Loss = 7.9514e-02, PNorm = 36.4989, GNorm = 0.6753, lr_0 = 7.4492e-04
Validation auc = 0.677465
Epoch 180
Validation auc = 0.670892
Epoch 181
Validation auc = 0.671362
Epoch 182
Validation auc = 0.692019
Epoch 183
Validation auc = 0.689671
Epoch 184
Loss = 3.0618e-02, PNorm = 36.5580, GNorm = 0.6912, lr_0 = 7.3855e-04
Validation auc = 0.671362
Epoch 185
Validation auc = 0.663380
Epoch 186
Validation auc = 0.671831
Epoch 187
Validation auc = 0.671362
Epoch 188
Validation auc = 0.670892
Epoch 189
Loss = 2.9744e-02, PNorm = 36.6194, GNorm = 0.3478, lr_0 = 7.3224e-04
Validation auc = 0.676995
Epoch 190
Validation auc = 0.678404
Epoch 191
Validation auc = 0.676056
Epoch 192
Validation auc = 0.670423
Epoch 193
Validation auc = 0.671831
Epoch 194
Loss = 1.7791e-02, PNorm = 36.6733, GNorm = 0.2418, lr_0 = 7.2598e-04
Validation auc = 0.677465
Epoch 195
Validation auc = 0.680751
Epoch 196
Validation auc = 0.673239
Epoch 197
Validation auc = 0.675587
Epoch 198
Validation auc = 0.687793
Epoch 199
Loss = 3.4040e-02, PNorm = 36.7236, GNorm = 1.1848, lr_0 = 7.1977e-04
Validation auc = 0.701408
Epoch 200
Validation auc = 0.692488
Epoch 201
Validation auc = 0.681221
Epoch 202
Validation auc = 0.686854
Epoch 203
Validation auc = 0.691549
Epoch 204
Loss = 3.8129e-02, PNorm = 36.7674, GNorm = 1.3003, lr_0 = 7.1362e-04
Validation auc = 0.672300
Epoch 205
Validation auc = 0.657746
Epoch 206
Validation auc = 0.676056
Epoch 207
Validation auc = 0.688732
Epoch 208
Validation auc = 0.689202
Epoch 209
Loss = 2.0085e-02, PNorm = 36.8184, GNorm = 0.2392, lr_0 = 7.0752e-04
Validation auc = 0.677934
Epoch 210
Validation auc = 0.679812
Epoch 211
Validation auc = 0.688263
Epoch 212
Validation auc = 0.688732
Epoch 213
Validation auc = 0.690141
Epoch 214
Loss = 1.9894e-02, PNorm = 36.8668, GNorm = 0.9050, lr_0 = 7.0147e-04
Validation auc = 0.687793
Epoch 215
Validation auc = 0.695775
Epoch 216
Validation auc = 0.697183
Epoch 217
Validation auc = 0.681221
Epoch 218
Validation auc = 0.675587
Epoch 219
Loss = 1.3199e-02, PNorm = 36.9145, GNorm = 0.1602, lr_0 = 6.9548e-04
Validation auc = 0.669014
Epoch 220
Validation auc = 0.668545
Epoch 221
Validation auc = 0.669953
Epoch 222
Validation auc = 0.680751
Epoch 223
Validation auc = 0.683099
Epoch 224
Loss = 2.4516e-02, PNorm = 36.9620, GNorm = 0.5766, lr_0 = 6.8953e-04
Validation auc = 0.691080
Epoch 225
Validation auc = 0.699061
Epoch 226
Validation auc = 0.692958
Epoch 227
Validation auc = 0.684977
Epoch 228
Validation auc = 0.685446
Epoch 229
Loss = 2.2788e-02, PNorm = 37.0069, GNorm = 1.2987, lr_0 = 6.8364e-04
Validation auc = 0.694836
Epoch 230
Validation auc = 0.705164
Epoch 231
Validation auc = 0.698592
Epoch 232
Validation auc = 0.681221
Epoch 233
Validation auc = 0.687324
Epoch 234
Loss = 1.3338e-02, PNorm = 37.0453, GNorm = 0.6658, lr_0 = 6.7779e-04
Validation auc = 0.686385
Epoch 235
Validation auc = 0.685915
Epoch 236
Validation auc = 0.680282
Epoch 237
Validation auc = 0.675587
Epoch 238
Validation auc = 0.685446
Epoch 239
Loss = 4.9159e-02, PNorm = 37.0826, GNorm = 0.9386, lr_0 = 6.7200e-04
Validation auc = 0.686385
Epoch 240
Validation auc = 0.680282
Epoch 241
Validation auc = 0.677465
Epoch 242
Validation auc = 0.681690
Epoch 243
Validation auc = 0.692958
Epoch 244
Loss = 2.3331e-02, PNorm = 37.1264, GNorm = 1.2354, lr_0 = 6.6625e-04
Validation auc = 0.692019
Epoch 245
Validation auc = 0.679343
Epoch 246
Validation auc = 0.671362
Epoch 247
Validation auc = 0.669484
Epoch 248
Validation auc = 0.668075
Epoch 249
Loss = 1.3144e-02, PNorm = 37.1637, GNorm = 0.4329, lr_0 = 6.6056e-04
Validation auc = 0.663380
Epoch 250
Validation auc = 0.666197
Epoch 251
Validation auc = 0.675587
Epoch 252
Validation auc = 0.681690
Epoch 253
Validation auc = 0.689671
Epoch 254
Loss = 2.1706e-02, PNorm = 37.2015, GNorm = 0.4925, lr_0 = 6.5491e-04
Validation auc = 0.690610
Epoch 255
Validation auc = 0.684038
Epoch 256
Validation auc = 0.677465
Epoch 257
Validation auc = 0.683568
Epoch 258
Validation auc = 0.693427
Epoch 259
Loss = 1.3819e-02, PNorm = 37.2296, GNorm = 0.4842, lr_0 = 6.4931e-04
Validation auc = 0.695775
Epoch 260
Validation auc = 0.692958
Epoch 261
Validation auc = 0.688263
Epoch 262
Validation auc = 0.681221
Epoch 263
Validation auc = 0.679812
Epoch 264
Loss = 3.1285e-02, PNorm = 37.2613, GNorm = 1.4260, lr_0 = 6.4376e-04
Validation auc = 0.696244
Epoch 265
Validation auc = 0.696714
Epoch 266
Validation auc = 0.675117
Epoch 267
Validation auc = 0.661972
Epoch 268
Validation auc = 0.671831
Epoch 269
Loss = 1.5436e-02, PNorm = 37.3021, GNorm = 0.9658, lr_0 = 6.3826e-04
Validation auc = 0.696714
Epoch 270
Validation auc = 0.700939
Epoch 271
Validation auc = 0.691080
Epoch 272
Validation auc = 0.687324
Epoch 273
Validation auc = 0.690610
Epoch 274
Loss = 1.1314e-02, PNorm = 37.3464, GNorm = 0.1421, lr_0 = 6.3280e-04
Validation auc = 0.698592
Epoch 275
Validation auc = 0.694836
Epoch 276
Validation auc = 0.687793
Epoch 277
Validation auc = 0.679343
Epoch 278
Validation auc = 0.678404
Epoch 279
Loss = 2.6298e-02, PNorm = 37.3806, GNorm = 1.3381, lr_0 = 6.2739e-04
Validation auc = 0.683568
Epoch 280
Validation auc = 0.692019
Epoch 281
Validation auc = 0.691080
Epoch 282
Validation auc = 0.683568
Epoch 283
Validation auc = 0.689671
Epoch 284
Loss = 1.7121e-02, PNorm = 37.4118, GNorm = 0.7674, lr_0 = 6.2203e-04
Validation auc = 0.693897
Epoch 285
Validation auc = 0.696714
Epoch 286
Validation auc = 0.684977
Epoch 287
Validation auc = 0.677465
Epoch 288
Validation auc = 0.672300
Epoch 289
Loss = 1.2331e-02, PNorm = 37.4404, GNorm = 0.1948, lr_0 = 6.1671e-04
Validation auc = 0.690610
Epoch 290
Validation auc = 0.700939
Epoch 291
Validation auc = 0.689671
Epoch 292
Validation auc = 0.679812
Epoch 293
Validation auc = 0.682160
Epoch 294
Loss = 4.1526e-02, PNorm = 37.4713, GNorm = 0.4133, lr_0 = 6.1144e-04
Validation auc = 0.703756
Epoch 295
Validation auc = 0.695305
Epoch 296
Validation auc = 0.680751
Epoch 297
Validation auc = 0.675117
Epoch 298
Validation auc = 0.680282
Epoch 299
Loss = 4.0388e-03, PNorm = 37.5039, GNorm = 0.1340, lr_0 = 6.0621e-04
Validation auc = 0.688732
Model 0 best validation auc = 0.723944 on epoch 38
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.904877
Ensemble test auc = 0.904877
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=400, bias=False)
        (W_h): Linear(in_features=400, out_features=400, bias=False)
        (W_o): Linear(in_features=533, out_features=400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=628, out_features=1, bias=True)
  )
)
Number of parameters = 433,029
Moving model to cuda
Epoch 0
Validation auc = 0.396244
Epoch 1
Validation auc = 0.413146
Epoch 2
Validation auc = 0.436150
Epoch 3
Validation auc = 0.460563
Epoch 4
Loss = 7.5306e-01, PNorm = 32.8319, GNorm = 1.4231, lr_0 = 6.5000e-04
Validation auc = 0.475117
Epoch 5
Validation auc = 0.521596
Epoch 6
Validation auc = 0.562911
Epoch 7
Validation auc = 0.636620
Epoch 8
Validation auc = 0.598122
Epoch 9
Loss = 6.0840e-01, PNorm = 32.9128, GNorm = 1.9330, lr_0 = 9.9743e-04
Validation auc = 0.631925
Epoch 10
Validation auc = 0.601408
Epoch 11
Validation auc = 0.592958
Epoch 12
Validation auc = 0.607981
Epoch 13
Validation auc = 0.624413
Epoch 14
Loss = 6.0034e-01, PNorm = 32.9789, GNorm = 1.3677, lr_0 = 9.8890e-04
Validation auc = 0.622535
Epoch 15
Validation auc = 0.626761
Epoch 16
Validation auc = 0.631925
Epoch 17
Validation auc = 0.642723
Epoch 18
Validation auc = 0.647418
Epoch 19
Loss = 4.6955e-01, PNorm = 33.0583, GNorm = 0.7576, lr_0 = 9.8045e-04
Validation auc = 0.631455
Epoch 20
Validation auc = 0.640376
Epoch 21
Validation auc = 0.655869
Epoch 22
Validation auc = 0.669484
Epoch 23
Validation auc = 0.653521
Epoch 24
Loss = 4.1963e-01, PNorm = 33.1565, GNorm = 0.9292, lr_0 = 9.7207e-04
Validation auc = 0.650704
Epoch 25
Validation auc = 0.684038
Epoch 26
Validation auc = 0.674178
Epoch 27
Validation auc = 0.715962
Epoch 28
Validation auc = 0.668075
Epoch 29
Loss = 4.4321e-01, PNorm = 33.2647, GNorm = 2.7901, lr_0 = 9.6376e-04
Validation auc = 0.673239
Epoch 30
Validation auc = 0.681221
Epoch 31
Validation auc = 0.647418
Epoch 32
Validation auc = 0.664789
Epoch 33
Validation auc = 0.703756
Epoch 34
Loss = 4.2510e-01, PNorm = 33.3807, GNorm = 1.0278, lr_0 = 9.5552e-04
Validation auc = 0.671362
Epoch 35
Validation auc = 0.681221
Epoch 36
Validation auc = 0.705164
Epoch 37
Validation auc = 0.688263
Epoch 38
Validation auc = 0.702817
Epoch 39
Loss = 4.5083e-01, PNorm = 33.5206, GNorm = 3.8759, lr_0 = 9.4735e-04
Validation auc = 0.701878
Epoch 40
Validation auc = 0.669484
Epoch 41
Validation auc = 0.700939
Epoch 42
Validation auc = 0.700939
Epoch 43
Validation auc = 0.666667
Epoch 44
Loss = 3.8998e-01, PNorm = 33.6647, GNorm = 1.7295, lr_0 = 9.3925e-04
Validation auc = 0.695305
Epoch 45
Validation auc = 0.696244
Epoch 46
Validation auc = 0.675587
Epoch 47
Validation auc = 0.716901
Epoch 48
Validation auc = 0.707042
Epoch 49
Loss = 3.1896e-01, PNorm = 33.8208, GNorm = 2.4809, lr_0 = 9.3122e-04
Validation auc = 0.706573
Epoch 50
Validation auc = 0.736150
Epoch 51
Validation auc = 0.688732
Epoch 52
Validation auc = 0.718779
Epoch 53
Validation auc = 0.718310
Epoch 54
Loss = 2.7878e-01, PNorm = 33.9855, GNorm = 2.1379, lr_0 = 9.2326e-04
Validation auc = 0.679343
Epoch 55
Validation auc = 0.721127
Epoch 56
Validation auc = 0.708451
Epoch 57
Validation auc = 0.698122
Epoch 58
Validation auc = 0.723944
Epoch 59
Loss = 2.4620e-01, PNorm = 34.1386, GNorm = 1.5336, lr_0 = 9.1537e-04
Validation auc = 0.705164
Epoch 60
Validation auc = 0.701408
Epoch 61
Validation auc = 0.728169
Epoch 62
Validation auc = 0.696714
Epoch 63
Validation auc = 0.715493
Epoch 64
Loss = 2.4541e-01, PNorm = 34.3162, GNorm = 3.3640, lr_0 = 9.0754e-04
Validation auc = 0.709859
Epoch 65
Validation auc = 0.711268
Epoch 66
Validation auc = 0.714085
Epoch 67
Validation auc = 0.687324
Epoch 68
Validation auc = 0.663850
Epoch 69
Loss = 2.2806e-01, PNorm = 34.4832, GNorm = 4.5640, lr_0 = 8.9978e-04
Validation auc = 0.660094
Epoch 70
Validation auc = 0.683099
Epoch 71
Validation auc = 0.726761
Epoch 72
Validation auc = 0.684507
Epoch 73
Validation auc = 0.715493
Epoch 74
Loss = 1.5546e-01, PNorm = 34.6372, GNorm = 1.1239, lr_0 = 8.9209e-04
Validation auc = 0.696244
Epoch 75
Validation auc = 0.696244
Epoch 76
Validation auc = 0.670892
Epoch 77
Validation auc = 0.654930
Epoch 78
Validation auc = 0.674648
Epoch 79
Loss = 2.1166e-01, PNorm = 34.7827, GNorm = 0.9383, lr_0 = 8.8447e-04
Validation auc = 0.683568
Epoch 80
Validation auc = 0.676995
Epoch 81
Validation auc = 0.692958
Epoch 82
Validation auc = 0.677465
Epoch 83
Validation auc = 0.681690
Epoch 84
Loss = 1.4183e-01, PNorm = 34.9168, GNorm = 1.3144, lr_0 = 8.7691e-04
Validation auc = 0.709390
Epoch 85
Validation auc = 0.692488
Epoch 86
Validation auc = 0.688263
Epoch 87
Validation auc = 0.684977
Epoch 88
Validation auc = 0.639437
Epoch 89
Loss = 2.1015e-01, PNorm = 35.0413, GNorm = 1.1173, lr_0 = 8.6941e-04
Validation auc = 0.653991
Epoch 90
Validation auc = 0.667136
Epoch 91
Validation auc = 0.685446
Epoch 92
Validation auc = 0.684507
Epoch 93
Validation auc = 0.692019
Epoch 94
Loss = 1.0334e-01, PNorm = 35.1711, GNorm = 1.3517, lr_0 = 8.6198e-04
Validation auc = 0.674178
Epoch 95
Validation auc = 0.678873
Epoch 96
Validation auc = 0.662911
Epoch 97
Validation auc = 0.684977
Epoch 98
Validation auc = 0.688732
Epoch 99
Loss = 1.2887e-01, PNorm = 35.2944, GNorm = 0.9646, lr_0 = 8.5461e-04
Validation auc = 0.700000
Epoch 100
Validation auc = 0.683568
Epoch 101
Validation auc = 0.679812
Epoch 102
Validation auc = 0.673709
Epoch 103
Validation auc = 0.660563
Epoch 104
Loss = 1.1591e-01, PNorm = 35.3998, GNorm = 0.8467, lr_0 = 8.4730e-04
Validation auc = 0.687324
Epoch 105
Validation auc = 0.661972
Epoch 106
Validation auc = 0.653521
Epoch 107
Validation auc = 0.678404
Epoch 108
Validation auc = 0.682629
Epoch 109
Loss = 8.4681e-02, PNorm = 35.5020, GNorm = 1.0911, lr_0 = 8.4006e-04
Validation auc = 0.688732
Epoch 110
Validation auc = 0.716432
Epoch 111
Validation auc = 0.690141
Epoch 112
Validation auc = 0.657746
Epoch 113
Validation auc = 0.682629
Epoch 114
Loss = 1.3923e-01, PNorm = 35.6080, GNorm = 1.6246, lr_0 = 8.3288e-04
Validation auc = 0.658685
Epoch 115
Validation auc = 0.651174
Epoch 116
Validation auc = 0.684507
Epoch 117
Validation auc = 0.669953
Epoch 118
Validation auc = 0.668545
Epoch 119
Loss = 3.6625e-02, PNorm = 35.7058, GNorm = 0.9217, lr_0 = 8.2576e-04
Validation auc = 0.679343
Epoch 120
Validation auc = 0.700469
Epoch 121
Validation auc = 0.687324
Epoch 122
Validation auc = 0.672300
Epoch 123
Validation auc = 0.671831
Epoch 124
Loss = 7.7230e-02, PNorm = 35.7892, GNorm = 1.9503, lr_0 = 8.1870e-04
Validation auc = 0.677934
Epoch 125
Validation auc = 0.660563
Epoch 126
Validation auc = 0.678873
Epoch 127
Validation auc = 0.677934
Epoch 128
Validation auc = 0.687793
Epoch 129
Loss = 3.6757e-02, PNorm = 35.8618, GNorm = 0.4532, lr_0 = 8.1170e-04
Validation auc = 0.677934
Epoch 130
Validation auc = 0.691549
Epoch 131
Validation auc = 0.690141
Epoch 132
Validation auc = 0.667606
Epoch 133
Validation auc = 0.656338
Epoch 134
Loss = 3.8802e-02, PNorm = 35.9289, GNorm = 1.9237, lr_0 = 8.0476e-04
Validation auc = 0.673239
Epoch 135
Validation auc = 0.684507
Epoch 136
Validation auc = 0.682629
Epoch 137
Validation auc = 0.680282
Epoch 138
Validation auc = 0.703286
Epoch 139
Loss = 6.4610e-02, PNorm = 35.9992, GNorm = 0.5212, lr_0 = 7.9788e-04
Validation auc = 0.647887
Epoch 140
Validation auc = 0.649765
Epoch 141
Validation auc = 0.649296
Epoch 142
Validation auc = 0.636150
Epoch 143
Validation auc = 0.657277
Epoch 144
Loss = 6.3169e-02, PNorm = 36.0735, GNorm = 1.5190, lr_0 = 7.9106e-04
Validation auc = 0.691080
Epoch 145
Validation auc = 0.684507
Epoch 146
Validation auc = 0.676995
Epoch 147
Validation auc = 0.689202
Epoch 148
Validation auc = 0.679812
Epoch 149
Loss = 3.9494e-02, PNorm = 36.1419, GNorm = 0.5022, lr_0 = 7.8430e-04
Validation auc = 0.665258
Epoch 150
Validation auc = 0.667136
Epoch 151
Validation auc = 0.692488
Epoch 152
Validation auc = 0.676995
Epoch 153
Validation auc = 0.667136
Epoch 154
Loss = 7.9499e-02, PNorm = 36.2162, GNorm = 0.3025, lr_0 = 7.7759e-04
Validation auc = 0.682629
Epoch 155
Validation auc = 0.691549
Epoch 156
Validation auc = 0.691080
Epoch 157
Validation auc = 0.681690
Epoch 158
Validation auc = 0.695305
Epoch 159
Loss = 4.5012e-02, PNorm = 36.2855, GNorm = 1.6198, lr_0 = 7.7095e-04
Validation auc = 0.690141
Epoch 160
Validation auc = 0.678404
Epoch 161
Validation auc = 0.670423
Epoch 162
Validation auc = 0.680282
Epoch 163
Validation auc = 0.664789
Epoch 164
Loss = 2.4993e-02, PNorm = 36.3444, GNorm = 0.2969, lr_0 = 7.6436e-04
Validation auc = 0.645070
Epoch 165
Validation auc = 0.641784
Epoch 166
Validation auc = 0.709390
Epoch 167
Validation auc = 0.639906
Epoch 168
Validation auc = 0.640376
Epoch 169
Loss = 1.1504e-01, PNorm = 36.4139, GNorm = 2.2514, lr_0 = 7.5782e-04
Validation auc = 0.670892
Epoch 170
Validation auc = 0.681221
Epoch 171
Validation auc = 0.681690
Epoch 172
Validation auc = 0.682160
Epoch 173
Validation auc = 0.694366
Epoch 174
Loss = 1.2817e-01, PNorm = 36.5114, GNorm = 3.9384, lr_0 = 7.5134e-04
Validation auc = 0.690610
Epoch 175
Validation auc = 0.676995
Epoch 176
Validation auc = 0.673239
Epoch 177
Validation auc = 0.686385
Epoch 178
Validation auc = 0.691080
Epoch 179
Loss = 8.3596e-02, PNorm = 36.5900, GNorm = 0.9691, lr_0 = 7.4492e-04
Validation auc = 0.677465
Epoch 180
Validation auc = 0.669014
Epoch 181
Validation auc = 0.672300
Epoch 182
Validation auc = 0.675587
Epoch 183
Validation auc = 0.681690
Epoch 184
Loss = 6.0515e-02, PNorm = 36.6604, GNorm = 0.9608, lr_0 = 7.3855e-04
Validation auc = 0.670423
Epoch 185
Validation auc = 0.659155
Epoch 186
Validation auc = 0.657277
Epoch 187
Validation auc = 0.673239
Epoch 188
Validation auc = 0.680282
Epoch 189
Loss = 6.5645e-02, PNorm = 36.7231, GNorm = 0.8165, lr_0 = 7.3224e-04
Validation auc = 0.670892
Epoch 190
Validation auc = 0.665728
Epoch 191
Validation auc = 0.678404
Epoch 192
Validation auc = 0.697653
Epoch 193
Validation auc = 0.689202
Epoch 194
Loss = 2.6024e-02, PNorm = 36.7910, GNorm = 0.3026, lr_0 = 7.2598e-04
Validation auc = 0.667136
Epoch 195
Validation auc = 0.660563
Epoch 196
Validation auc = 0.665258
Epoch 197
Validation auc = 0.675587
Epoch 198
Validation auc = 0.683568
Epoch 199
Loss = 3.5694e-02, PNorm = 36.8511, GNorm = 0.5519, lr_0 = 7.1977e-04
Validation auc = 0.680751
Epoch 200
Validation auc = 0.681690
Epoch 201
Validation auc = 0.687324
Epoch 202
Validation auc = 0.693897
Epoch 203
Validation auc = 0.695305
Epoch 204
Loss = 8.1828e-02, PNorm = 36.8937, GNorm = 1.1312, lr_0 = 7.1362e-04
Validation auc = 0.700469
Epoch 205
Validation auc = 0.692488
Epoch 206
Validation auc = 0.675117
Epoch 207
Validation auc = 0.668545
Epoch 208
Validation auc = 0.663850
Epoch 209
Loss = 1.5316e-02, PNorm = 36.9324, GNorm = 0.5269, lr_0 = 7.0752e-04
Validation auc = 0.670892
Epoch 210
Validation auc = 0.675117
Epoch 211
Validation auc = 0.672300
Epoch 212
Validation auc = 0.676995
Epoch 213
Validation auc = 0.675117
Epoch 214
Loss = 2.9827e-02, PNorm = 36.9721, GNorm = 0.3460, lr_0 = 7.0147e-04
Validation auc = 0.683568
Epoch 215
Validation auc = 0.685915
Epoch 216
Validation auc = 0.684977
Epoch 217
Validation auc = 0.675587
Epoch 218
Validation auc = 0.667606
Epoch 219
Loss = 1.4745e-02, PNorm = 37.0082, GNorm = 0.3318, lr_0 = 6.9548e-04
Validation auc = 0.665728
Epoch 220
Validation auc = 0.665728
Epoch 221
Validation auc = 0.665728
Epoch 222
Validation auc = 0.663850
Epoch 223
Validation auc = 0.659624
Epoch 224
Loss = 3.9877e-02, PNorm = 37.0438, GNorm = 0.3927, lr_0 = 6.8953e-04
Validation auc = 0.663850
Epoch 225
Validation auc = 0.669953
Epoch 226
Validation auc = 0.661502
Epoch 227
Validation auc = 0.655399
Epoch 228
Validation auc = 0.668075
Epoch 229
Loss = 9.5398e-03, PNorm = 37.0741, GNorm = 0.3607, lr_0 = 6.8364e-04
Validation auc = 0.679343
Epoch 230
Validation auc = 0.683568
Epoch 231
Validation auc = 0.681221
Epoch 232
Validation auc = 0.682160
Epoch 233
Validation auc = 0.681221
Epoch 234
Loss = 1.0798e-02, PNorm = 37.1015, GNorm = 0.3781, lr_0 = 6.7779e-04
Validation auc = 0.685446
Epoch 235
Validation auc = 0.682160
Epoch 236
Validation auc = 0.679343
Epoch 237
Validation auc = 0.679343
Epoch 238
Validation auc = 0.684038
Epoch 239
Loss = 1.3970e-02, PNorm = 37.1268, GNorm = 0.6631, lr_0 = 6.7200e-04
Validation auc = 0.690141
Epoch 240
Validation auc = 0.683568
Epoch 241
Validation auc = 0.670892
Epoch 242
Validation auc = 0.671831
Epoch 243
Validation auc = 0.682160
Epoch 244
Loss = 4.1742e-02, PNorm = 37.1538, GNorm = 3.4533, lr_0 = 6.6625e-04
Validation auc = 0.670423
Epoch 245
Validation auc = 0.654460
Epoch 246
Validation auc = 0.683099
Epoch 247
Validation auc = 0.701878
Epoch 248
Validation auc = 0.690141
Epoch 249
Loss = 2.3887e-02, PNorm = 37.1828, GNorm = 0.7557, lr_0 = 6.6056e-04
Validation auc = 0.666197
Epoch 250
Validation auc = 0.663380
Epoch 251
Validation auc = 0.660563
Epoch 252
Validation auc = 0.647418
Epoch 253
Validation auc = 0.649765
Epoch 254
Loss = 1.4158e-02, PNorm = 37.2283, GNorm = 0.1856, lr_0 = 6.5491e-04
Validation auc = 0.661502
Epoch 255
Validation auc = 0.667606
Epoch 256
Validation auc = 0.671831
Epoch 257
Validation auc = 0.673709
Epoch 258
Validation auc = 0.676056
Epoch 259
Loss = 2.0943e-02, PNorm = 37.2711, GNorm = 0.5457, lr_0 = 6.4931e-04
Validation auc = 0.676056
Epoch 260
Validation auc = 0.668545
Epoch 261
Validation auc = 0.668545
Epoch 262
Validation auc = 0.667136
Epoch 263
Validation auc = 0.664789
Epoch 264
Loss = 8.9994e-03, PNorm = 37.3100, GNorm = 0.1834, lr_0 = 6.4376e-04
Validation auc = 0.656808
Epoch 265
Validation auc = 0.656338
Epoch 266
Validation auc = 0.667606
Epoch 267
Validation auc = 0.672770
Epoch 268
Validation auc = 0.667136
Epoch 269
Loss = 1.0573e-02, PNorm = 37.3444, GNorm = 0.3272, lr_0 = 6.3826e-04
Validation auc = 0.651174
Epoch 270
Validation auc = 0.654460
Epoch 271
Validation auc = 0.669014
Epoch 272
Validation auc = 0.670892
Epoch 273
Validation auc = 0.658685
Epoch 274
Loss = 1.9564e-02, PNorm = 37.3811, GNorm = 1.3032, lr_0 = 6.3280e-04
Validation auc = 0.651174
Epoch 275
Validation auc = 0.656808
Epoch 276
Validation auc = 0.677934
Epoch 277
Validation auc = 0.671362
Epoch 278
Validation auc = 0.662911
Epoch 279
Loss = 1.2144e-02, PNorm = 37.4185, GNorm = 1.1641, lr_0 = 6.2739e-04
Validation auc = 0.659155
Epoch 280
Validation auc = 0.660563
Epoch 281
Validation auc = 0.662911
Epoch 282
Validation auc = 0.664319
Epoch 283
Validation auc = 0.661502
Epoch 284
Loss = 1.2537e-02, PNorm = 37.4516, GNorm = 0.3572, lr_0 = 6.2203e-04
Validation auc = 0.661502
Epoch 285
Validation auc = 0.660563
Epoch 286
Validation auc = 0.665728
Epoch 287
Validation auc = 0.677934
Epoch 288
Validation auc = 0.680282
Epoch 289
Loss = 5.7939e-03, PNorm = 37.4811, GNorm = 0.2826, lr_0 = 6.1671e-04
Validation auc = 0.680282
Epoch 290
Validation auc = 0.675587
Epoch 291
Validation auc = 0.672770
Epoch 292
Validation auc = 0.685446
Epoch 293
Validation auc = 0.688732
Epoch 294
Loss = 2.1626e-02, PNorm = 37.5078, GNorm = 0.6880, lr_0 = 6.1144e-04
Validation auc = 0.673709
Epoch 295
Validation auc = 0.661502
Epoch 296
Validation auc = 0.659624
Epoch 297
Validation auc = 0.671831
Epoch 298
Validation auc = 0.677934
Epoch 299
Loss = 1.5392e-02, PNorm = 37.5370, GNorm = 0.1727, lr_0 = 6.0621e-04
Validation auc = 0.674648
Model 0 best validation auc = 0.736150 on epoch 50
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test auc = 0.894642
Ensemble test auc = 0.894642
10-fold cross validation
	Seed 0 ==> test auc = 0.904275
	Seed 1 ==> test auc = 0.880795
	Seed 2 ==> test auc = 0.877784
	Seed 3 ==> test auc = 0.915111
	Seed 4 ==> test auc = 0.888621
	Seed 5 ==> test auc = 0.873570
	Seed 6 ==> test auc = 0.916918
	Seed 7 ==> test auc = 0.875376
	Seed 8 ==> test auc = 0.904877
	Seed 9 ==> test auc = 0.894642
Overall test auc = 0.893197 +/- 0.015554
Elapsed time = 0:05:05
